summaryrefslogtreecommitdiff
path: root/git-compat-util.h
AgeCommit message (Collapse)AuthorFilesLines
2021-12-10Merge branch 'cb/mingw-gmtime-r'Libravatar Junio C Hamano1-1/+3
Build fix on Windows. * cb/mingw-gmtime-r: mingw: avoid fallback for {local,gm}time_r()
2021-12-10Merge branch 'bc/require-c99'Libravatar Junio C Hamano1-0/+13
Weather balloon to break people with compilers that do not support C99. * bc/require-c99: git-compat-util: add a test balloon for C99 support
2021-12-01git-compat-util: add a test balloon for C99 supportLibravatar brian m. carlson1-0/+13
The C99 standard was released in January 1999, now 22 years ago. It provides a variety of useful features, including variadic arguments for macros, declarations after statements, designated initializers, and a wide variety of other useful features, many of which we already use. We'd like to take advantage of these features, but we want to be cautious. As far as we know, all major compilers now support C99 or a later C standard, such as C11 or C17. POSIX has required C99 support as a requirement for the 2001 revision, so we can safely assume any POSIX system which we are interested in supporting has C99. Even MSVC, long a holdout against modern C, now supports both C11 and C17 with an appropriate update. Moreover, even if people are using an older version of MSVC on these systems, they will generally need some implementation of the standard Unix utilities for the testsuite, and GNU coreutils, the most common option, has required C99 since 2009. Therefore, we can safely assume that a suitable version of GCC or clang is available to users even if their version of MSVC is not sufficiently capable. Let's add a test balloon to git-compat-util.h to see if anyone is using an older compiler. We'll add a comment telling people how to enable this functionality on GCC and Clang, even though modern versions of both will automatically do the right thing, and ask people still experiencing a problem to report that to us on the list. Note that C89 compilers don't provide the __STDC_VERSION__ macro, so we use a well-known hack of using "- 0". On compilers with this macro, it doesn't change the value, and on C89 compilers, the macro will be replaced with nothing, and our value will be 0. For sparse, we explicitly request the gnu99 style because we've traditionally taken advantage of some GCC- and clang-specific extensions when available and we'd like to retain the ability to do that. sparse also defaults to C89 without it, so things will fail for us if we don't. Update the cmake configuration to require C11 for MSVC. We do this because this will make MSVC to use C11, since it does not explicitly support C99. We do this with a compiler options because setting the C_STANDARD option does not work in our CI on MSVC and at the moment, we don't want to require C11 for Unix compilers. In the Makefile, don't set any compiler flags for the compiler itself, since on some systems, such as FreeBSD, we actually need C11, and asking for C99 causes things to fail to compile. The error message should make it obvious what's going wrong and allow a user to set the appropriate option when building in the event they're using a Unix compiler that doesn't support it by default. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-29Merge branch 'mc/clean-smudge-with-llp64'Libravatar Junio C Hamano1-0/+25
The clean/smudge conversion code path has been prepared to better work on platforms where ulong is narrower than size_t. * mc/clean-smudge-with-llp64: clean/smudge: allow clean filters to process extremely large files odb: guard against data loss checking out a huge file git-compat-util: introduce more size_t helpers odb: teach read_blob_entry to use size_t t1051: introduce a smudge filter test for extremely large files test-lib: add prerequisite for 64-bit platforms test-tool genzeros: generate large amounts of data more efficiently test-genzeros: allow more than 2G zeros in Windows
2021-11-29Merge branch 'jc/unsetenv-returns-an-int'Libravatar Junio C Hamano1-1/+1
The compatibility implementation for unsetenv(3) were written to mimic ancient, non-POSIX, variant seen in an old glibc; it has been changed to return an integer to match the more modern era. * jc/unsetenv-returns-an-int: unsetenv(3) returns int, not void
2021-11-27mingw: avoid fallback for {local,gm}time_r()Libravatar Carlo Marcelo Arenas Belón1-1/+3
mingw-w64's pthread_unistd.h had a bug that mistakenly (because there is no support for the *lockfile() functions required[1]) defined _POSIX_THREAD_SAFE_FUNCTIONS and that was being worked around since 3ecd153a3b (compat/mingw: support MSys2-based MinGW build, 2016-01-14). The bug was fixed in winphtreads, but as a side effect, leaves the reentrant functions from time.h no longer visible and therefore breaks the build. Since the intention all along was to avoid using the fallback functions, formalize the use of POSIX by setting the corresponding feature flag and compile out the implementation for the fallback functions. [1] https://unix.org/whitepapers/reentrant.html Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-03git-compat-util: introduce more size_t helpersLibravatar Johannes Schindelin1-0/+25
We will use them in the next commit. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29unsetenv(3) returns int, not voidLibravatar Junio C Hamano1-1/+1
This compatilibity implementation has been returning a wrong type, ever since 731043fd (Add compat/unsetenv.c ., 2006-01-25) added to the system, yet nobody noticed it in the past 16 years, presumably because no code checks failures in their unsetenv() calls. Sigh. For now, make it always succeed. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29wrapper: remove xunsetenv()Libravatar Carlo Marcelo Arenas Belón1-1/+0
Remove the unused wrapper function. Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-06Merge branch 'ab/repo-settings-cleanup'Libravatar Junio C Hamano1-0/+2
Code cleanup. * ab/repo-settings-cleanup: repository.h: don't use a mix of int and bitfields repo-settings.c: simplify the setup read-cache & fetch-negotiator: check "enum" values in switch() environment.c: remove test-specific "ignore_untracked..." variable wrapper.c: add x{un,}setenv(), and use xsetenv() in environment.c
2021-09-23Merge branch 'cb/unix-sockets-with-windows'Libravatar Junio C Hamano1-0/+3
Adjust credential-cache helper to Windows. * cb/unix-sockets-with-windows: git-compat-util: include declaration for unix sockets in windows credential-cache: check for windows specific errors t0301: fixes for windows compatibility
2021-09-22wrapper.c: add x{un,}setenv(), and use xsetenv() in environment.cLibravatar Ævar Arnfjörð Bjarmason1-0/+2
Add fatal wrappers for setenv() and unsetenv(). In d7ac12b25d3 (Add set_git_dir() function, 2007-08-01) we started checking its return value, and since 48988c4d0c3 (set_git_dir: die when setenv() fails, 2018-03-30) we've had set_git_dir_1() die if we couldn't set it. Let's provide a wrapper for both, this will be useful in many other places, a subsequent patch will make another use of xsetenv(). The checking of the return value here is over-eager according to setenv(3) and POSIX. It's documented as returning just -1 or 0, so perhaps we should be checking -1 explicitly. Let's just instead die on any non-zero, if our C library is so broken as to return something else than -1 on error (and perhaps not set errno?) the worst we'll do is die with a nonsensical errno value, but we'll want to die in either case. Let's make these return "void" instead of "int". As far as I can tell there's no other x*() wrappers that needed to make the decision of deviating from the signature in the C library, but since their return value is only used to indicate errors (so we'd die here), we can catch unreachable code such as if (xsetenv(...) < 0) [...]; I think it would be OK skip the NULL check of the "name" here for the calls to die_errno(). Almost all of our setenv() callers are taking a constant string hardcoded in the source as the first argument, and for the rest we can probably assume they've done the NULL check themselves. Even if they didn't, modern C libraries are forgiving about it (e.g. glibc formatting it as "(null)"), on those that aren't, well, we were about to die anyway. But let's include the check anyway for good measure. 1. https://pubs.opengroup.org/onlinepubs/009604499/functions/setenv.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-14git-compat-util: include declaration for unix sockets in windowsLibravatar Carlo Marcelo Arenas Belón1-0/+3
Available since Windows 10 release 1803 and Windows Server 2019. NO_UNIX_SOCKETS is still the default for Windows builds, as they need to keep backward compatibility with releases up to Windows 7, but allow including the header otherwise. Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-03gettext: remove optional non-standard parens in N_() definitionLibravatar Ævar Arnfjörð Bjarmason1-4/+0
Remove the USE_PARENS_AROUND_GETTEXT_N compile-time option which was meant to catch an inadvertent mistake which is too obscure to maintain this facility. The backstory of how USE_PARENS_AROUND_GETTEXT_N came about is: When I added the N_() macro in 65784830366 (i18n: add no-op _() and N_() wrappers, 2011-02-22) it was defined as: #define N_(msgid) (msgid) This is non-standard C, as was noticed and fixed in 642f85faab2 (i18n: avoid parenthesized string as array initializer, 2011-04-07). I.e. this needed to be defined as: #define N_(msgid) msgid Then in e62cd35a3e8 (i18n: log: mark parseopt strings for translation, 2012-08-20) when "builtin_log_usage" was marked for translation the string concatenation for passing to usage() added in 1c370ea4e51 (Show usage string for 'git log -h', 'git show -h' and 'git diff -h', 2009-08-06) was faithfully preserved: - "git log [<options>] [<since>..<until>] [[--] <path>...]\n" - " or: git show [options] <object>...", + N_("git log [<options>] [<since>..<until>] [[--] <path>...]\n") + N_(" or: git show [options] <object>..."), This was then fixed to be the expected array of usage strings in e66dc0cc4b1 (log.c: fix translation markings, 2015-01-06) rather than a string with multiple "\n"-delimited usage strings, and finally in 290c8e7a3fe (gettext.h: add parentheses around N_ expansion if supported, 2015-01-11) USE_PARENS_AROUND_GETTEXT_N was added to ensure this mistake didn't happen again. I think that even if this was a N_()-specific issue this USE_PARENS_AROUND_GETTEXT_N facility wouldn't be worth it, the issue would be too rare to worry about. But I also think that 290c8e7a3fe which introduced USE_PARENS_AROUND_GETTEXT_N misattributed the problem. The issue wasn't with the N_() macro added in e62cd35a3e8, but that before the N_() macro existed in the codebase the initial migration to parse_options() in 1c370ea4e51 continued passsing in a "\n"-delimited string, when the new API it was migrating to supported and expected the passing of an array. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-16Merge branch 'ew/mmap-failures'Libravatar Junio C Hamano1-0/+1
Error message update. * ew/mmap-failures: xmmap: inform Linux users of tuning knobs on ENOMEM
2021-07-08Merge branch 'ar/typofix'Libravatar Junio C Hamano1-1/+1
Typofixes. * ar/typofix: *: fix typos which duplicate a word
2021-06-29xmmap: inform Linux users of tuning knobs on ENOMEMLibravatar Eric Wong1-0/+1
Linux users may benefit from additional information on how to avoid ENOMEM from mmap despite the system having enough RAM to accomodate them. We can't reliably unmap pack windows to work around the issue since malloc and other library routines may mmap without our knowledge. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-14*: fix typos which duplicate a wordLibravatar Andrei Rybak1-1/+1
Fix typos in documentation, code comments, and RelNotes which repeat various words. In trivial cases, just delete the duplicated word and rewrap text, if needed. Reword the affected sentence in Documentation/RelNotes/1.8.4.txt for it to make sense. Signed-off-by: Andrei Rybak <rybak.a.v@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-10Merge branch 'jn/size-t-casted-to-off-t-fix'Libravatar Junio C Hamano1-4/+2
Rewrite code that triggers undefined behaiour warning. * jn/size-t-casted-to-off-t-fix: xsize_t: avoid implementation defined behavior when len < 0
2021-05-19xsize_t: avoid implementation defined behavior when len < 0Libravatar Jonathan Nieder1-4/+2
The xsize_t helper aims to safely convert an off_t to a size_t, erroring out when a file offset is too large to fit into a memory address. It does this by using two casts: size_t size = (size_t) len; if (len != (off_t) size) ... error out ... On a platform with sizeof(size_t) < sizeof(off_t), this check is safe and correct. The first cast truncates to a size_t by finding the remainder modulo SIZE_MAX+1 (see C99 section 6.3.1.3 Signed and unsigned integers) and the second promotes to an off_t, meaning the result is true if and only if len is representable as a size_t. On other platforms, this two-casts strategy still works well (always succeeds) for len >= 0. But for len < 0, when the first cast succeeds and produces SIZE_MAX + 1 + len, the resulting value is too large to be represented as an off_t, so the second cast produces implementation defined behavior. In practice, it is likely to produce a result of true despite len not being representable as size_t. Simplify by replacing with a more straightforward check: compare len to the relevant bounds and then cast it. (To avoid a -Wsign-compare warning, after checking that len >= 0, we explicitly convert to a sufficiently-large unsigned type before comparing to SIZE_MAX.) In practice, this is not likely to come up since typical callers use nonnegative len. Still, it's helpful to handle this case to make the behavior easy to reason about. Historical note: the original bounds-checking in 46be82dfd0 (xsize_t: check whether we lose bits, 2010-07-28) did not produce this implementation-defined behavior, though it still did not handle negative offsets. It was not until 73560c793a (git-compat-util.h: xsize_t() - avoid -Wsign-compare warnings, 2017-09-21) introduced the double cast that the implementation-defined behavior was triggered. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-13Merge branch 'tb/precompose-prefix-simplify'Libravatar Junio C Hamano1-0/+5
Streamline the codepath to fix the UTF-8 encoding issues in the argv[] and the prefix on macOS. * tb/precompose-prefix-simplify: macOS: precompose startup_info->prefix precompose_utf8: make precompose_string_if_needed() public
2021-04-05precompose_utf8: make precompose_string_if_needed() publicLibravatar Torsten Bögershausen1-0/+5
commit 5c327502 (MacOS: precompose_argv_prefix(), 2021-02-03) uses the function precompose_string_if_needed() internally. It is only used from precompose_argv_prefix() and therefore static in compat/precompose_utf8.c Expose this function, it will be used in the next commit. While there, allow passing a NULL pointer, which will return NULL. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-22Merge branch 'jk/open-dotgitx-with-nofollow'Libravatar Junio C Hamano1-0/+7
It does not make sense to make ".gitattributes", ".gitignore" and ".mailmap" symlinks, as they are supposed to be usable from the object store (think: bare repositories where HEAD:.mailmap etc. are used). When these files are symbolic links, we used to read the contents of the files pointed by them by mistake, which has been corrected. * jk/open-dotgitx-with-nofollow: mailmap: do not respect symlinks for in-tree .mailmap exclude: do not respect symlinks for in-tree .gitignore attr: do not respect symlinks for in-tree .gitattributes exclude: add flags parameter to add_patterns() attr: convert "macro_ok" into a flags field add open_nofollow() helper
2021-03-13git-compat-util.h: drop trailing semicolon from macro definitionLibravatar René Scharfe1-1/+1
Make CALLOC_ARRAY usable like a function by requiring callers to supply the trailing semicolon, which all of the current ones already do. With the extra semicolon e.g. the following code wouldn't compile because it disconnects the "else" from the "if": if (condition) CALLOC_ARRAY(ptr, n); else whatever(); Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-08Sync with Git 2.30.2 for CVE-2021-21300Libravatar Junio C Hamano1-0/+5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-04Merge branch 'jk/open-returns-eintr'Libravatar Junio C Hamano1-0/+6
Work around platforms whose open() is reported to return EINTR (it shouldn't, as we do our signals with SA_RESTART). * jk/open-returns-eintr: config.mak.uname: enable OPEN_RETURNS_EINTR for macOS Big Sur Makefile: add OPEN_RETURNS_EINTR knob
2021-02-26Makefile: add OPEN_RETURNS_EINTR knobLibravatar Jeff King1-0/+6
On some platforms, open() reportedly returns EINTR when opening regular files and we receive a signal (usually SIGALRM from our progress meter). This shouldn't happen, as open() should be a restartable syscall, and we specify SA_RESTART when setting up the alarm handler. So it may actually be a kernel or libc bug for this to happen. But it has been reported on at least one version of Linux (on a network filesystem): https://lore.kernel.org/git/c8061cce-71e4-17bd-a56a-a5fed93804da@neanderfunk.de/ as well as on macOS starting with Big Sur even on a regular filesystem. We can work around it by retrying open() calls that get EINTR, just as we do for read(), etc. Since we don't ever _want_ to interrupt an open() call, we can get away with just redefining open, rather than insisting all callsites use xopen(). We actually do have an xopen() wrapper already (and it even does this retry, though there's no indication of it being an observed problem back then; it seems simply to have been lifted from xread(), etc). But it is used hardly anywhere, and isn't suitable for general use because it will die() on error. In theory we could combine the two, but it's awkward to do so because of the variable-args interface of open(). This patch adds a Makefile knob for enabling the workaround. It's not enabled by default for any platforms in config.mak.uname yet, as we don't have enough data to decide how common this is (I have not been able to reproduce on either Linux or Big Sur myself). It may be worth enabling preemptively anyway, since the cost is pretty low (if we don't see an EINTR, it's just an extra conditional). However, note that we must not enable this on Windows. It doesn't do anything there, and the macro overrides the existing mingw_open() redirection. I've added a preemptive #undef here in the mingw header (which is processed first) to just quietly disable it (we could also make it an #error, but there is little point in being so aggressive). Reported-by: Aleksey Kliger <alklig@microsoft.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-16add open_nofollow() helperLibravatar Jeff King1-0/+7
Some callers of open() would like to use O_NOFOLLOW, but it is not available on all platforms. Let's abstract this into a helper function so we can provide system-specific implementations. Some light web-searching reveals that we might be able to get something similar on Windows using FILE_FLAG_OPEN_REPARSE_POINT. I didn't dig into this further. For other systems without O_NOFOLLOW or any equivalent, we have two options for fallback: - we can just open anyway, following symlinks; this may have security implications (e.g., following untrusted in-tree symlinks) - we can determine whether the path is a symlink with lstat(). This is slower (two syscalls instead of one), but that may be acceptable for infrequent uses like looking up .gitattributes files (especially because we can get away with a single syscall for the common case of ENOENT). It's also racy, but should be sufficient for our needs (we are worried about in-tree symlinks that we ourselves would have previously created). We could make it non-racy at the cost of making it even slower, by doing an fstat() on the opened descriptor and comparing the dev/ino fields to the original lstat(). This patch implements the lstat() option in its slightly-faster racy form. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-12Merge branch 'tb/precompose-prefix-too'Libravatar Junio C Hamano1-2/+2
When commands are started from a subdirectory, they may have to compare the path to the subdirectory (called prefix and found out from $(pwd)) with the tracked paths. On macOS, $(pwd) and readdir() yield decomposed path, while the tracked paths are usually normalized to the precomposed form, causing mismatch. This has been fixed by taking the same approach used to normalize the command line arguments. * tb/precompose-prefix-too: MacOS: precompose_argv_prefix()
2021-02-12Sync with 2.29.3Libravatar Johannes Schindelin1-0/+5
* maint-2.29: Git 2.29.3 Git 2.28.1 Git 2.27.1 Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.28.1Libravatar Johannes Schindelin1-0/+5
* maint-2.28: Git 2.28.1 Git 2.27.1 Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.27.1Libravatar Johannes Schindelin1-0/+5
* maint-2.27: Git 2.27.1 Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.26.3Libravatar Johannes Schindelin1-0/+5
* maint-2.26: Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.24.4Libravatar Johannes Schindelin1-0/+5
* maint-2.24: Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.23.4Libravatar Johannes Schindelin1-0/+5
* maint-2.23: Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.22.5Libravatar Johannes Schindelin1-0/+5
* maint-2.22: Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.21.4Libravatar Johannes Schindelin1-0/+5
* maint-2.21: Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.20.5Libravatar Johannes Schindelin1-0/+5
* maint-2.20: Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.19.6Libravatar Johannes Schindelin1-0/+5
* maint-2.19: Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.18.5Libravatar Johannes Schindelin1-0/+5
* maint-2.18: Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.17.6Libravatar Johannes Schindelin1-0/+5
* maint-2.17: Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12checkout: fix bug that makes checkout follow symlinks in leading pathLibravatar Matheus Tavares1-0/+5
Before checking out a file, we have to confirm that all of its leading components are real existing directories. And to reduce the number of lstat() calls in this process, we cache the last leading path known to contain only directories. However, when a path collision occurs (e.g. when checking out case-sensitive files in case-insensitive file systems), a cached path might have its file type changed on disk, leaving the cache on an invalid state. Normally, this doesn't bring any bad consequences as we usually check out files in index order, and therefore, by the time the cached path becomes outdated, we no longer need it anyway (because all files in that directory would have already been written). But, there are some users of the checkout machinery that do not always follow the index order. In particular: checkout-index writes the paths in the same order that they appear on the CLI (or stdin); and the delayed checkout feature -- used when a long-running filter process replies with "status=delayed" -- postpones the checkout of some entries, thus modifying the checkout order. When we have to check out an out-of-order entry and the lstat() cache is invalid (due to a previous path collision), checkout_entry() may end up using the invalid data and thrusting that the leading components are real directories when, in reality, they are not. In the best case scenario, where the directory was replaced by a regular file, the user will get an error: "fatal: unable to create file 'foo/bar': Not a directory". But if the directory was replaced by a symlink, checkout could actually end up following the symlink and writing the file at a wrong place, even outside the repository. Since delayed checkout is affected by this bug, it could be used by an attacker to write arbitrary files during the clone of a maliciously crafted repository. Some candidate solutions considered were to disable the lstat() cache during unordered checkouts or sort the entries before passing them to the checkout machinery. But both ideas include some performance penalty and they don't future-proof the code against new unordered use cases. Instead, we now manually reset the lstat cache whenever we successfully remove a directory. Note: We are not even checking whether the directory was the same as the lstat cache points to because we might face a scenario where the paths refer to the same location but differ due to case folding, precomposed UTF-8 issues, or the presence of `..` components in the path. Two regression tests, with case-collisions and utf8-collisions, are also added for both checkout-index and delayed checkout. Note: to make the previously mentioned clone attack unfeasible, it would be sufficient to reset the lstat cache only after the remove_subtree() call inside checkout_entry(). This is the place where we would remove a directory whose path collides with the path of another entry that we are currently trying to check out (possibly a symlink). However, in the interest of a thorough fix that does not leave Git open to similar-but-not-identical attack vectors, we decided to intercept all `rmdir()` calls in one fell swoop. This addresses CVE-2021-21300. Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
2021-02-03MacOS: precompose_argv_prefix()Libravatar Torsten Bögershausen1-2/+2
The following sequence leads to a "BUG" assertion running under MacOS: DIR=git-test-restore-p Adiarnfd=$(printf 'A\314\210') DIRNAME=xx${Adiarnfd}yy mkdir $DIR && cd $DIR && git init && mkdir $DIRNAME && cd $DIRNAME && echo "Initial" >file && git add file && echo "One more line" >>file && echo y | git restore -p . Initialized empty Git repository in /tmp/git-test-restore-p/.git/ BUG: pathspec.c:495: error initializing pathspec_item Cannot close git diff-index --cached --numstat [snip] The command `git restore` is run from a directory inside a Git repo. Git needs to split the $CWD into 2 parts: The path to the repo and "the rest", if any. "The rest" becomes a "prefix" later used inside the pathspec code. As an example, "/path/to/repo/dir-inside-repå" would determine "/path/to/repo" as the root of the repo, the place where the configuration file .git/config is found. The rest becomes the prefix ("dir-inside-repå"), from where the pathspec machinery expands the ".", more about this later. If there is a decomposed form, (making the decomposing visible like this), "dir-inside-rep°a" doesn't match "dir-inside-repå". Git commands need to: (a) read the configuration variable "core.precomposeunicode" (b) precocompose argv[] (c) precompose the prefix, if there was any The first commit, 76759c7dff53 "git on Mac OS and precomposed unicode" addressed (a) and (b). The call to precompose_argv() was added into parse-options.c, because that seemed to be a good place when the patch was written. Commands that don't use parse-options need to do (a) and (b) themselfs. The commands `diff-files`, `diff-index`, `diff-tree` and `diff` learned (a) and (b) in commit 90a78b83e0b8 "diff: run arguments through precompose_argv" Branch names (or refs in general) using decomposed code points resulting in decomposed file names had been fixed in commit 8e712ef6fc97 "Honor core.precomposeUnicode in more places" The bug report from above shows 2 things: - more commands need to handle precomposed unicode - (c) should be implemented for all commands using pathspecs Solution: precompose_argv() now handles the prefix (if needed), and is renamed into precompose_argv_prefix(). Inside this function the config variable core.precomposeunicode is read into the global variable precomposed_unicode, as before. This reading is skipped if precomposed_unicode had been read before. The original patch for preocomposed unicode, 76759c7dff53, placed precompose_argv() into parse-options.c Now add it into git.c::run_builtin() as well. Existing precompose calls in diff-files.c and others may become redundant, and if we audit the callflows that reach these places to make sure that they can never be reached without going through the new call added to run_builtin(), we might be able to remove these existing ones. But in this commit, we do not bother to do so and leave these precompose callsites as they are. Because precompose() is idempotent and can be called on an already precomposed string safely, this is safer than removing existing calls without fully vetting the callflows. There is certainly room for cleanups - this change intends to be a bug fix. Cleanups needs more tests in e.g. t/t3910-mac-os-precompose.sh, and should be done in future commits. [1] git-bugreport-2021-01-06-1209.txt (git can't deal with special characters) [2] https://lore.kernel.org/git/A102844A-9501-4A86-854D-E3B387D378AA@icloud.com/ Reported-by: Daniel Troger <random_n0body@icloud.com> Helped-By: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-27git-compat-util: always enable variadic macrosLibravatar Jeff King1-2/+5
We allow variadic macros in the code base, but only if there is fallback code for platforms that lack it. This leads to some annoyances: - the code is more complicated because of the fallbacks (e.g., trace_printf(), etc, is implemented twice with a set of parallel wrappers). - some constructs are just impossible and we've had to live without them (e.g., a cross between FLEX_ALLOC and xstrfmt) Since this feature is present in C99, we may be able to start counting on it being available everywhere. Let's start with a weather balloon patch to find out. This patch makes the absolute minimal change by always setting HAVE_VARIADIC_MACROS. If somebody runs into a platform where it's a problem, they can undo it by commenting out the define. Likewise, if we have to revert this, it would be quite unlikely to cause conflicts. Once we feel comfortable that this is the right direction, then we can start ripping out all the spots that actually look at the flag, and removing the dead code. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-12-18Merge branch 'jc/compat-util-setitimer-fix'Libravatar Junio C Hamano1-1/+1
Fix a recent bug in a rarely used replacement code. * jc/compat-util-setitimer-fix: compat-util: pretend that stub setitimer() always succeeds
2020-12-15compat-util: pretend that stub setitimer() always succeedsLibravatar Junio C Hamano1-1/+1
When 15b52a44 (compat-util: type-check parameters of no-op replacement functions, 2020-08-06) turned a handful of no-op C-preprocessor macros into static inline functions to give the callers a better type checking for their parameters, it forgot to return anything from the stubbed out setitimer() function, even though the function was defined to return an int just like the real thing. Since the original C-preprocessor macro implementation was to just turn the call to the function an empty statement, we know that the existing callers do not check the return value from it, and it does not matter what value we return. But it is safer to pretend that the call succeeded by returning 0 than making it fail by returning -1 and clobbering errno with some value. Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-30Merge branch 'hn/sleep-millisec-decl'Libravatar Junio C Hamano1-0/+2
Move a definition of compatibility wrapper from cache.h to git-compat-util.h * hn/sleep-millisec-decl: move sleep_millisec to git-compat-util.h
2020-11-24move sleep_millisec to git-compat-util.hLibravatar Han-Wen Nienhuys1-0/+2
The sleep function is defined in wrapper.c, so it makes more sense to be a in system compatibility header. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-02Merge branch 'jk/report-fn-typedef'Libravatar Junio C Hamano1-5/+7
Code clean-up. * jk/report-fn-typedef: usage: define a type for a reporting function
2020-10-16usage: define a type for a reporting functionLibravatar Jeff King1-5/+7
The usage, die, warning, and error routines all work with a function pointer that takes the message to be reported. We usually just mention the function's full type inline. But this makes the use of these pointers hard to read, especially because C's syntax for returning a function pointer is so awful: void (*get_error_routine(void))(const char *err, va_list params); Unless you read it very carefully, this looks like a function pointer declaration. Let's instead use a single typedef to define a reporting function, which is the same for all four types. Note that this also removes the "extern" from these declarations to match the surrounding functions. They were missed in 554544276a (*.[ch]: remove extern from function declarations using spatch, 2019-04-29) presumably because of the unusual syntax. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>