summaryrefslogtreecommitdiff
path: root/strbuf.c
AgeCommit message (Collapse)AuthorFilesLines
2018-05-30Merge branch 'js/use-bug-macro'Libravatar Junio C Hamano1-2/+2
Developer support update, by using BUG() macro instead of die() to mark codepaths that should not happen more clearly. * js/use-bug-macro: BUG_exit_code: fix sparse "symbol not declared" warning Convert remaining die*(BUG) messages Replace all die("BUG: ...") calls by BUG() ones run-command: use BUG() to report bugs, not die() test-tool: help verifying BUG() code paths
2018-05-23Merge branch 'en/rename-directory-detection-reboot'Libravatar Junio C Hamano1-0/+16
Rename detection logic in "diff" family that is used in "merge" has learned to guess when all of x/a, x/b and x/c have moved to z/a, z/b and z/c, it is likely that x/d added in the meantime would also want to move to z/d by taking the hint that the entire directory 'x' moved to 'z'. A bug causing dirty files involved in a rename to be overwritten during merge has also been fixed as part of this work. Incidentally, this also avoids updating a file in the working tree after a (non-trivial) merge whose result matches what our side originally had. * en/rename-directory-detection-reboot: (36 commits) merge-recursive: fix check for skipability of working tree updates merge-recursive: make "Auto-merging" comment show for other merges merge-recursive: fix remainder of was_dirty() to use original index merge-recursive: fix was_tracked() to quit lying with some renamed paths t6046: testcases checking whether updates can be skipped in a merge merge-recursive: avoid triggering add_cacheinfo error with dirty mod merge-recursive: move more is_dirty handling to merge_content merge-recursive: improve add_cacheinfo error handling merge-recursive: avoid spurious rename/rename conflict from dir renames directory rename detection: new testcases showcasing a pair of bugs merge-recursive: fix remaining directory rename + dirty overwrite cases merge-recursive: fix overwriting dirty files involved in renames merge-recursive: avoid clobbering untracked files with directory renames merge-recursive: apply necessary modifications for directory renames merge-recursive: when comparing files, don't include trees merge-recursive: check for file level conflicts then get new name merge-recursive: add computation of collisions due to dir rename & merging merge-recursive: check for directory level conflicts merge-recursive: add get_directory_renames() merge-recursive: make a helper function for cleanup for handle_renames ...
2018-05-08merge-recursive: check for file level conflicts then get new nameLibravatar Elijah Newren1-0/+16
Before trying to apply directory renames to paths within the given directories, we want to make sure that there aren't conflicts at the file level either. If there aren't any, then get the new name from any directory renames. Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-08Merge branch 'ls/checkout-encoding'Libravatar Junio C Hamano1-1/+21
The new "checkout-encoding" attribute can ask Git to convert the contents to the specified encoding when checking out to the working tree (and the other way around when checking in). * ls/checkout-encoding: convert: add round trip check based on 'core.checkRoundtripEncoding' convert: add tracing for 'working-tree-encoding' attribute convert: check for detectable errors in UTF encodings convert: add 'working-tree-encoding' attribute utf8: add function to detect a missing UTF-16/32 BOM utf8: add function to detect prohibited UTF-16/32 BOM utf8: teach same_encoding() alternative UTF encoding names strbuf: add a case insensitive starts_with() strbuf: add xstrdup_toupper() strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
2018-05-06Replace all die("BUG: ...") calls by BUG() onesLibravatar Johannes Schindelin1-2/+2
In d8193743e08 (usage.c: add BUG() function, 2017-05-12), a new macro was introduced to use for reporting bugs instead of die(). It was then subsequently used to convert one single caller in 588a538ae55 (setup_git_env: convert die("BUG") to BUG(), 2017-05-12). The cover letter of the patch series containing this patch (cf 20170513032414.mfrwabt4hovujde2@sigill.intra.peff.net) is not terribly clear why only one call site was converted, or what the plan is for other, similar calls to die() to report bugs. Let's just convert all remaining ones in one fell swoop. This trick was performed by this invocation: sed -i 's/die("BUG: /BUG("/g' $(git grep -l 'die("BUG' \*.c) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11Revert "Merge branch 'en/rename-directory-detection'"Libravatar Junio C Hamano1-16/+0
This reverts commit e4bb62fa1eeee689744b413e29a50b4d1dae6886, reversing changes made to 468165c1d8a442994a825f3684528361727cd8c0. The topic appears to inflict severe regression in renaming merges, even though the promise of it was that it would improve them. We do not yet know which exact change in the topic was wrong, but in the meantime, let's play it safe and revert it out of 'master' before real Git-using projects are harmed. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-10Merge branch 'bc/object-id'Libravatar Junio C Hamano1-2/+2
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (36 commits) convert: convert to struct object_id sha1_file: introduce a constant for max header length Convert lookup_replace_object to struct object_id sha1_file: convert read_sha1_file to struct object_id sha1_file: convert read_object_with_reference to object_id tree-walk: convert tree entry functions to object_id streaming: convert istream internals to struct object_id tree-walk: convert get_tree_entry_follow_symlinks internals to object_id builtin/notes: convert static functions to object_id builtin/fmt-merge-msg: convert remaining code to object_id sha1_file: convert sha1_object_info* to object_id Convert remaining callers of sha1_object_info_extended to object_id packfile: convert unpack_entry to struct object_id sha1_file: convert retry_bad_packed_offset to struct object_id sha1_file: convert assert_sha1_type to object_id builtin/mktree: convert to struct object_id streaming: convert open_istream to use struct object_id sha1_file: convert check_sha1_signature to struct object_id sha1_file: convert read_loose_object to use struct object_id builtin/index-pack: convert struct ref_delta_entry to object_id ...
2018-04-10Merge branch 'en/rename-directory-detection'Libravatar Junio C Hamano1-0/+16
Rename detection logic in "diff" family that is used in "merge" has learned to guess when all of x/a, x/b and x/c have moved to z/a, z/b and z/c, it is likely that x/d added in the meantime would also want to move to z/d by taking the hint that the entire directory 'x' moved to 'z'. A bug causing dirty files involved in a rename to be overwritten during merge has also been fixed as part of this work. * en/rename-directory-detection: (29 commits) merge-recursive: ensure we write updates for directory-renamed file merge-recursive: avoid spurious rename/rename conflict from dir renames directory rename detection: new testcases showcasing a pair of bugs merge-recursive: fix remaining directory rename + dirty overwrite cases merge-recursive: fix overwriting dirty files involved in renames merge-recursive: avoid clobbering untracked files with directory renames merge-recursive: apply necessary modifications for directory renames merge-recursive: when comparing files, don't include trees merge-recursive: check for file level conflicts then get new name merge-recursive: add computation of collisions due to dir rename & merging merge-recursive: check for directory level conflicts merge-recursive: add get_directory_renames() merge-recursive: make a helper function for cleanup for handle_renames merge-recursive: split out code for determining diff_filepairs merge-recursive: make !o->detect_rename codepath more obvious merge-recursive: fix leaks of allocated renames and diff_filepairs merge-recursive: introduce new functions to handle rename logic merge-recursive: move the get_renames() function directory rename detection: tests for handling overwriting dirty files directory rename detection: tests for handling overwriting untracked files ...
2018-03-14Merge branch 'nd/worktree-move'Libravatar Junio C Hamano1-0/+8
"git worktree" learned move and remove subcommands. * nd/worktree-move: t2028: fix minor error and issues in newly-added "worktree move" tests worktree remove: allow it when $GIT_WORK_TREE is already gone worktree remove: new command worktree move: refuse to move worktrees with submodules worktree move: accept destination as directory worktree move: new command worktree.c: add update_worktree_location() worktree.c: add validate_worktree()
2018-03-14Convert find_unique_abbrev* to struct object_idLibravatar brian m. carlson1-1/+1
Convert find_unique_abbrev and find_unique_abbrev_r to each take a pointer to struct object_id. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14strbuf: convert strbuf_add_unique_abbrev to use struct object_idLibravatar brian m. carlson1-2/+2
Convert the declaration and definition of strbuf_add_unique_abbrev to make it take a pointer to struct object_id. Predeclare the struct in strbuf.h, as cache.h includes strbuf.h before it declares the struct, and otherwise the struct declaration would have the wrong scope. Apply the following semantic patch, along with the standard object_id transforms, to adjust the callers: @@ expression E1, E2, E3; @@ - strbuf_add_unique_abbrev(E1, E2.hash, E3); + strbuf_add_unique_abbrev(E1, &E2, E3); @@ expression E1, E2, E3; @@ - strbuf_add_unique_abbrev(E1, E2->hash, E3); + strbuf_add_unique_abbrev(E1, E2, E3); Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-09strbuf: add a case insensitive starts_with()Libravatar Lars Schneider1-0/+9
Check in a case insensitive manner if one string is a prefix of another string. This function is used in a subsequent commit. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-23strbuf_read_file(): preserve errno across close() callLibravatar Jeff King1-1/+5
If we encounter a read error, the user may want to report it by looking at errno. However, our close() call may clobber errno, leading to confusing results. Let's save and restore it in the error case. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-15strbuf: add xstrdup_toupper()Libravatar Lars Schneider1-0/+12
Create a copy of an existing string and make all characters upper case. Similar xstrdup_tolower(). This function is used in a subsequent commit. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-15strbuf: remove unnecessary NUL assignment in xstrdup_tolower()Libravatar Lars Schneider1-1/+0
Since 3733e69464 (use xmallocz to avoid size arithmetic, 2016-02-22) we allocate the buffer for the lower case string with xmallocz(). This already ensures a NUL at the end of the allocated buffer. Remove the unnecessary assignment. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-14merge-recursive: check for file level conflicts then get new nameLibravatar Elijah Newren1-0/+16
Before trying to apply directory renames to paths within the given directories, we want to make sure that there aren't conflicts at the file level either. If there aren't any, then get the new name from any directory renames. Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-12worktree move: accept destination as directoryLibravatar Nguyễn Thái Ngọc Duy1-0/+8
Similar to "mv a b/", which is actually "mv a b/a", we extract basename of source worktree and create a directory of the same name at destination if dst path is a directory. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05Merge branch 'jd/fix-strbuf-add-urlencode-bytes'Libravatar Junio C Hamano1-1/+1
Bytes with high-bit set were encoded incorrectly and made credential helper fail. * jd/fix-strbuf-add-urlencode-bytes: strbuf: fix urlencode format string on signed char
2017-12-28Merge branch 'cc/skip-to-optional-val'Libravatar Junio C Hamano1-0/+22
Introduce a helper to simplify code to parse a common pattern that expects either "--key" or "--key=<something>". * cc/skip-to-optional-val: t4045: reindent to make helpers readable diff: add tests for --relative without optional prefix value diff: use skip_to_optional_arg_default() in parsing --relative diff: use skip_to_optional_arg_default() diff: use skip_to_optional_arg() index-pack: use skip_to_optional_arg() git-compat-util: introduce skip_to_optional_arg()
2017-12-27Merge branch 'rs/strbuf-read-once-reset-length'Libravatar Junio C Hamano1-0/+3
Leakfix. * rs/strbuf-read-once-reset-length: strbuf: release memory on read error in strbuf_read_once()
2017-12-22strbuf: fix urlencode format string on signed charLibravatar Julien Dusser1-1/+1
Git credential fails with special char in password with remote: Invalid username or password. fatal: Authentication failed for File ~/.git-credential contains badly urlencoded characters %ffffffXX%ffffffYY instead of %XX%YY. Add a cast to an unsigned char to fix urlencode use of %02x on a char. Signed-off-by: Julien Dusser <julien.dusser@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-11git-compat-util: introduce skip_to_optional_arg()Libravatar Christian Couder1-0/+22
We often accept both a "--key" option and a "--key=<val>" option. These options currently are parsed using something like: if (!strcmp(arg, "--key")) { /* do something */ } else if (skip_prefix(arg, "--key=", &arg)) { /* do something with arg */ } which is a bit cumbersome compared to just: if (skip_to_optional_arg(arg, "--key", &arg)) { /* do something with arg */ } This also introduces skip_to_optional_arg_default() for the few cases where something different should be done when the first argument is exactly "--key" than when it is exactly "--key=". In general it is better for UI consistency and simplicity if "--key" and "--key=" do the same thing though, so that using skip_to_optional_arg() should be encouraged compared to skip_to_optional_arg_default(). Note that these functions can be used to parse any "key=value" string where "key" is also considered as valid, not just command line options. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-07strbuf: release memory on read error in strbuf_read_once()Libravatar René Scharfe1-0/+3
If other strbuf add functions cause the first allocation and subsequently encounter an error then they release the memory, restoring the pristine state of the strbuf. That simplifies error handling for callers. Do the same in strbuf_read_once(), and do it also in case no bytes were read -- which may or may not be an error as well, depending on the caller. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-22Merge branch 'rs/strbuf-getwholeline-fix'Libravatar Junio C Hamano1-0/+1
A helper function to read a single whole line into strbuf mistakenly triggered OOM error at EOF under certain conditions, which has been fixed. * rs/strbuf-getwholeline-fix: strbuf: clear errno before calling getdelim(3)
2017-08-10strbuf: clear errno before calling getdelim(3)Libravatar René Scharfe1-0/+1
getdelim(3) returns -1 at the end of the file and if it encounters an error, but sets errno only in the latter case. Set errno to zero before calling it to avoid misdiagnosing an out-of-memory condition due to a left-over value from some other function call. Reported-by: Yaroslav Halchenko <yoh@onerussian.com> Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-06Merge branch 'ab/strbuf-addftime-tzname-boolify'Libravatar Junio C Hamano1-3/+2
strbuf_addftime() is further getting tweaked. * ab/strbuf-addftime-tzname-boolify: strbuf: change an always NULL/"" strbuf_addftime() param to bool strbuf.h comment: discuss strbuf_addftime() arguments in order
2017-07-01strbuf: change an always NULL/"" strbuf_addftime() param to boolLibravatar Ævar Arnfjörð Bjarmason1-3/+2
strbuf_addftime() allows callers to pass a time zone name for expanding %Z. The only current caller either passes the empty string or NULL, in which case %Z is handed over verbatim to strftime(3). Replace that string parameter with a flag controlling whether to remove %Z from the format specification. This simplifies the code. Commit-message-by: René Scharfe <l.s.r@web.de> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-24Merge branch 'rs/pretty-add-again'Libravatar Junio C Hamano1-7/+0
The pretty-format specifiers like '%h', '%t', etc. had an optimization that no longer works correctly. In preparation/hope of getting it correctly implemented, first discard the optimization that is broken. * rs/pretty-add-again: pretty: recalculate duplicate short hashes
2017-06-15strbuf: let strbuf_addftime handle %z and %Z itselfLibravatar René Scharfe1-4/+37
There is no portable way to pass timezone information to strftime. Add parameters for timezone offset and name to strbuf_addftime and let it handle the timezone-related format specifiers %z and %Z internally. Callers can opt out for %Z by passing NULL as timezone name. %z is always handled internally -- this helps on Windows, where strftime would expand it to a timezone name (same as %Z), in violation of POSIX. Modifiers are not handled, e.g. %Ez is still passed to strftime. Use an empty string as timezone name in show_date (the only current caller) for now because we only have the timezone offset in non-local mode. POSIX allows %Z to resolve to an empty string in case of missing information. Helped-by: Ulrich Mueller <ulm@gentoo.org> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15pretty: recalculate duplicate short hashesLibravatar René Scharfe1-7/+0
b9c6232138 (--format=pretty: avoid calculating expensive expansions twice) optimized adding short hashes multiple times by using the fact that the output strbuf was only ever simply appended to and copying the added string from the previous run. That prerequisite is no longer given; we now have modfiers like %< and %+ that can cause the cache to lose track of the correct offsets. Remove it. Reported-by: Michael Giuffrida <michaelpg@chromium.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-30Merge branch 'rs/freebsd-getcwd-workaround'Libravatar Junio C Hamano1-0/+11
FreeBSD implementation of getcwd(3) behaved differently when an intermediate directory is unreadable/unsearchable depending on the length of the buffer provided, which our strbuf_getcwd() was not aware of. strbuf_getcwd() has been taught to cope with it better. * rs/freebsd-getcwd-workaround: strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD
2017-03-26strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSDLibravatar René Scharfe1-0/+11
FreeBSD implements getcwd(3) as a syscall, but falls back to a version based on readdir(3) if it fails for some reason. The latter requires permissions to read and execute path components, while the former does not. That means that if our buffer is too small and we're missing rights we could get EACCES, but we may succeed with a bigger buffer. Keep retrying if getcwd(3) indicates lack of permissions until our buffer can fit PATH_MAX bytes, as that's the maximum supported by the syscall on FreeBSD anyway. This way we do what we can to be able to benefit from the syscall, but we also won't loop forever if there is a real permission issue. This fixes a regression introduced with 7333ed17 (setup: convert setup_git_directory_gently_1 et al. to strbuf, 2014-07-28) for paths longer than 127 bytes with components that miss read or execute permissions (e.g. 0711 on /home for privacy reasons); we used a fixed PATH_MAX-sized buffer before. Reported-by: Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-27strbuf: add strbuf_add_real_path()Libravatar René Scharfe1-0/+11
Add a function for appending the canonized absolute pathname of a given path to a strbuf. It keeps the existing contents intact, as expected of a function of the strbuf_add() family, while avoiding copying the result if the given strbuf is empty. It's more consistent with the rest of the strbuf API than strbuf_realpath(), which it's wrapping. Also add a semantic patch demonstrating its intended usage and apply it to the current tree. Using strbuf_add_real_path() instead of calling strbuf_addstr() and real_path() avoids an extra copy to a static buffer. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10link_alt_odb_entry: handle normalize_path errorsLibravatar Jeff King1-0/+20
When we add a new alternate to the list, we try to normalize out any redundant "..", etc. However, we do not look at the return value of normalize_path_copy(), and will happily continue with a path that could not be normalized. Worse, the normalizing process is done in-place, so we are left with whatever half-finished working state the normalizing function was in. Fortunately, this cannot cause us to read past the end of our buffer, as that working state will always leave the NUL from the original path in place. And we do tend to notice problems when we check is_directory() on the path. But you can see the nonsense that we feed to is_directory with an entry like: this/../../is/../../way/../../too/../../deep/../../to/../../resolve in your objects/info/alternates, which yields: error: object directory /to/e/deep/too/way//ects/this/../../is/../../way/../../too/../../deep/../../to/../../resolve does not exist; check .git/objects/info/alternates. We can easily fix this just by checking the return value. But that makes it hard to generate a good error message, since we're normalizing in-place and our input value has been overwritten by cruft. Instead, let's provide a strbuf helper that does an in-place normalize, but restores the original contents on error. This uses a second buffer under the hood, which is slightly less efficient, but this is not a performance-critical code path. The strbuf helper can also properly set the "len" parameter of the strbuf before returning. Just doing: normalize_path_copy(buf.buf, buf.buf); will shorten the string, but leave buf.len at the original length. That may be confusing to later code which uses the strbuf. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-29Merge branch 'rs/strbuf-remove-fix' into maintLibravatar Junio C Hamano1-1/+1
Code cleanup. * rs/strbuf-remove-fix: strbuf: use valid pointer in strbuf_remove()
2016-09-21Merge branch 'rs/strbuf-remove-fix'Libravatar Junio C Hamano1-1/+1
Code cleanup. * rs/strbuf-remove-fix: strbuf: use valid pointer in strbuf_remove()
2016-09-13strbuf: use valid pointer in strbuf_remove()Libravatar René Scharfe1-1/+1
The fourth argument of strbuf_splice() is passed to memcpy(3), which is not supposed to handle NULL pointers. Let's be extra careful and use a valid empty string instead. It even shortens the source code. :) Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-08Merge branch 'rs/use-strbuf-addbuf' into maintLibravatar Junio C Hamano1-0/+7
Code cleanup. * rs/use-strbuf-addbuf: strbuf: avoid calling strbuf_grow() twice in strbuf_addbuf() use strbuf_addbuf() for appending a strbuf to another
2016-07-25Merge branch 'rs/use-strbuf-addbuf'Libravatar Junio C Hamano1-0/+7
Code cleanup. * rs/use-strbuf-addbuf: strbuf: avoid calling strbuf_grow() twice in strbuf_addbuf() use strbuf_addbuf() for appending a strbuf to another
2016-07-22strbuf: avoid calling strbuf_grow() twice in strbuf_addbuf()Libravatar René Scharfe1-0/+7
Implement strbuf_addbuf() as a normal function in order to avoid calling strbuf_grow() twice, with the second callinside strbud_add() being a no-op. This is slightly faster and also reduces the text size a bit. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-14Merge branch 'jk/getwholeline-getdelim-empty' into maintLibravatar Junio C Hamano1-1/+7
strbuf_getwholeline() did not NUL-terminate the buffer on certain corner cases in its error codepath. * jk/getwholeline-getdelim-empty: strbuf_getwholeline: NUL-terminate getdelim buffer on error
2016-04-06Merge branch 'sb/submodule-parallel-update'Libravatar Junio C Hamano1-0/+6
A major part of "git submodule update" has been ported to C to take advantage of the recently added framework to run download tasks in parallel. * sb/submodule-parallel-update: clone: allow an explicit argument for parallel submodule clones submodule update: expose parallelism to the user submodule helper: remove double 'fatal: ' prefix git submodule update: have a dedicated helper for cloning run_processes_parallel: rename parameters for the callbacks run_processes_parallel: treat output of children as byte array submodule update: direct error message to stderr fetching submodules: respect `submodule.fetchJobs` config option submodule-config: drop check against NULL submodule-config: keep update strategy around
2016-04-03Merge branch 'jk/getwholeline-getdelim-empty'Libravatar Junio C Hamano1-1/+7
strbuf_getwholeline() did not NUL-terminate the buffer on certain corner cases in its error codepath. * jk/getwholeline-getdelim-empty: strbuf_getwholeline: NUL-terminate getdelim buffer on error
2016-03-05strbuf_getwholeline: NUL-terminate getdelim buffer on errorLibravatar Jeff King1-1/+7
Commit 0cc30e0 (strbuf_getwholeline: use getdelim if it is available, 2015-04-16) tries to clean up after getdelim() returns EOF, but gets one case wrong, which can lead in some obscure cases to us reading uninitialized memory. After getdelim() returns -1, we re-initialize the strbuf only if sb->buf is NULL. The thinking was that either: 1. We fed an existing allocated buffer to getdelim(), and at most it would have realloc'd, leaving our NUL in place. 2. We didn't have a buffer to feed, so we gave getdelim() NULL; sb->buf will remain NULL, and we just want to restore the empty slopbuf. But that second case isn't quite right. getdelim() may allocate a buffer, write nothing into it, and then return EOF. The resulting strbuf rightfully has sb->len set to "0", but is missing the NUL terminator in the first byte. Most call-sites are fine with this. They see the EOF and don't bother looking at the strbuf. Or they notice that sb->len is empty, and don't look at the contents. But there's at least one case that does neither, and relies on parsing the resulting (possibly zero-length) string: fast-import. You can see this in action with the new test (though we probably only notice failure there when run with --valgrind or ASAN). We can fix this by unconditionally resetting the strbuf when we have a buffer after getdelim(). That fixes case 2 above. Case 1 is probably already fine in practice, but it does not hurt for us to re-assert our invariants (especially because we are relying on whatever getdelim() happens to do, which may vary from platform to platform). Our fix covers that case, too. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01run_processes_parallel: treat output of children as byte arrayLibravatar Stefan Beller1-0/+6
We do not want the output to be interrupted by a NUL byte, so we cannot use raw fputs. Introduce strbuf_write to avoid having long arguments in run-command.c. Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-26Merge branch 'jk/tighten-alloc'Libravatar Junio C Hamano1-1/+1
Update various codepaths to avoid manually-counted malloc(). * jk/tighten-alloc: (22 commits) ewah: convert to REALLOC_ARRAY, etc convert ewah/bitmap code to use xmalloc diff_populate_gitlink: use a strbuf transport_anonymize_url: use xstrfmt git-compat-util: drop mempcpy compat code sequencer: simplify memory allocation of get_message test-path-utils: fix normalize_path_copy output buffer size fetch-pack: simplify add_sought_entry fast-import: simplify allocation in start_packfile write_untracked_extension: use FLEX_ALLOC helper prepare_{git,shell}_cmd: use argv_array use st_add and st_mult for allocation size computation convert trivial cases to FLEX_ARRAY macros use xmallocz to avoid size arithmetic convert trivial cases to ALLOC_ARRAY convert manual allocations to argv_array argv-array: add detach function add helpers for allocating flex-array structs harden REALLOC_ARRAY and xcalloc against size_t overflow tree-diff: catch integer overflow in combine_diff_path allocation ...
2016-02-22use xmallocz to avoid size arithmeticLibravatar Jeff King1-1/+1
We frequently allocate strings as xmalloc(len + 1), where the extra 1 is for the NUL terminator. This can be done more simply with xmallocz, which also checks for integer overflow. There's no case where switching xmalloc(n+1) to xmallocz(n) is wrong; the result is the same length, and malloc made no guarantees about what was in the buffer anyway. But in some cases, we can stop manually placing NUL at the end of the allocated buffer. But that's only safe if it's clear that the contents will always fill the buffer. In each case where this patch does so, I manually examined the control flow, and I tried to err on the side of caution. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-28Merge branch 'jc/strbuf-getline'Libravatar Junio C Hamano1-3/+25
The preliminary clean-up for jc/peace-with-crlf topic. * jc/strbuf-getline: strbuf: give strbuf_getline() to the "most text friendly" variant checkout-index: there are only two possible line terminations update-index: there are only two possible line terminations check-ignore: there are only two possible line terminations check-attr: there are only two possible line terminations mktree: there are only two possible line terminations strbuf: introduce strbuf_getline_{lf,nul}() strbuf: make strbuf_getline_crlf() global strbuf: miniscule style fix
2016-01-15strbuf: give strbuf_getline() to the "most text friendly" variantLibravatar Junio C Hamano1-4/+4
Now there is no direct caller to strbuf_getline(), we can demote it to file-scope static that is private to strbuf.c and rename it to strbuf_getdelim(). Rename strbuf_getline_crlf(), which is designed to be the most "text friendly" variant, and allow it to take over this simplest name, strbuf_getline(), so we can add more uses of it without having to type _crlf over and over again in the coming steps. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-15strbuf: introduce strbuf_getline_{lf,nul}()Libravatar Junio C Hamano1-0/+10
The strbuf_getline() interface allows a byte other than LF or NUL as the line terminator, but this is only because I wrote these codepaths anticipating that there might be a value other than NUL and LF that could be useful when I introduced line_termination long time ago. No useful caller that uses other value has emerged. By now, it is clear that the interface is overly broad without a good reason. Many codepaths have hardcoded preference to read either LF terminated or NUL terminated records from their input, and then call strbuf_getline() with LF or NUL as the third parameter. This step introduces two thin wrappers around strbuf_getline(), namely, strbuf_getline_lf() and strbuf_getline_nul(), and mechanically rewrites these call sites to call either one of them. The changes contained in this patch are: * introduction of these two functions in strbuf.[ch] * mechanical conversion of all callers to strbuf_getline() with either '\n' or '\0' as the third parameter to instead call the respective thin wrapper. After this step, output from "git grep 'strbuf_getline('" would become a lot smaller. An interim goal of this series is to make this an empty set, so that we can have strbuf_getline_crlf() take over the shorter name strbuf_getline(). Signed-off-by: Junio C Hamano <gitster@pobox.com>