summaryrefslogtreecommitdiff
path: root/git-compat-util.h
AgeCommit message (Collapse)AuthorFilesLines
2022-02-21C99: remove hardcoded-out !HAVE_VARIADIC_MACROS codeLibravatar Ævar Arnfjörð Bjarmason1-12/+0
Remove the "else" branches of the HAVE_VARIADIC_MACROS macro, which have been unconditionally omitted since 765dc168882 (git-compat-util: always enable variadic macros, 2021-01-28). Since were always omitted, anyone trying to use a compiler without variadic macro support to compile a git since version git v2.31.0 or later would have had a compilation error. 10 months across a few releases since then should have been enough time for anyone who cared to run into that and report the issue. In addition to that, for anyone unsetting HAVE_VARIADIC_MACROS we've been emitting extremely verbose warnings since at least ee4512ed481 (trace2: create new combined trace facility, 2019-02-22). That's because there is no such thing as a "region_enter_printf" or "region_leave_printf" format, so at least under GCC and Clang everything that includes trace.h (almost every file) emits a couple of warnings about that. There's a large benefit to being able to have a hard dependency rely on variadic macros, the code surrounding usage.c is hard to maintain if we need to write two implementations of everything, and by relying on "__FILE__" and "__LINE__" along with "__VA_ARGS__" we can in the future make error(), die() etc. log where they were called from. We've also recently merged d67fc4bf0ba (Merge branch 'bc/require-c99', 2021-12-10) which further cements our hard dependency on C99. So let's delete the fallback code, and update our CodingGuidelines to note that we depend on this. The added bullet-point starts with lower-case for consistency with other bullet-points in that section. The diff in "trace.h" is relatively hard to read, since we need to retain the existing API docs, which were comments on the code used if HAVE_VARIADIC_MACROS was not defined. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-21git-compat-util.h: clarify GCC v.s. C99-specific in commentLibravatar Ævar Arnfjörð Bjarmason1-3/+1
Change a comment added in e208f9cc757 (make error()'s constant return value more visible, 2012-12-15). It's not correct that this is GCC-ism anymore, it's code that uses standard C99 features. The comment being changed here pre-dates the HAVE_VARIADIC_MACROS define, which we got in e05bed960d3 (trace: add 'file:line' to all trace output, 2014-07-12). The original implementation of an error() macro) in e208f9cc757 used a GCC-ism with the paste operator (see the commit message for mention of it), but that was dropped later by 9798f7e5f9 (Use __VA_ARGS__ for all of error's arguments, 2013-02-08), giving us the C99-portable version we have now. While we could remove the __GNUC__ define here, it might cause issues for other compilers or static analysis systems, so let's not. See 87fe5df365 (inline constant return from error() function, 2014-05-06) for one such issue. See also e05bed960d3 (trace: add 'file:line' to all trace output, 2014-07-12) for another comment about GNUC's handling of __VA_ARGS__. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-16Merge branch 'ab/auto-detect-zlib-compress2'Libravatar Junio C Hamano1-0/+12
The build procedure has been taught to notice older version of zlib and enable our replacement uncompress2() automatically. * ab/auto-detect-zlib-compress2: compat: auto-detect if zlib has uncompress2()
2022-02-11Merge branch 'bc/csprng-mktemps'Libravatar Junio C Hamano1-0/+19
Pick a better random number generator and use it when we prepare temporary filenames. * bc/csprng-mktemps: wrapper: use a CSPRNG to generate random file names wrapper: add a helper to generate numbers from a CSPRNG
2022-01-26compat: auto-detect if zlib has uncompress2()Libravatar Ævar Arnfjörð Bjarmason1-0/+12
We have a copy of uncompress2() implementation in compat/ so that we can build with an older version of zlib that lack the function, and the build procedure selects if it is used via the NO_UNCOMPRESS2 $(MAKE) variable. This is yet another "annoying" knob the porters need to tweak on platforms that are not common enough to have the default set in the config.mak.uname file. Attempt to instead ask the system header <zlib.h> to decide if we need the compatibility implementation. This is a deviation from the way we have been handling the "compatiblity" features so far, and if it can be done cleanly enough, it could work as a model for features that need compatibility definition we discover in the future. With that goal in mind, avoid expedient but ugly hacks, like shoving the code that is conditionally compiled into an unrelated .c file, which may not work in future cases---instead, take an approach that uses a file that is independently compiled and stands on its own. Compile and link compat/zlib-uncompress2.c file unconditionally, but conditionally hide the implementation behind #if/#endif when zlib version is 1.2.9 or newer, and unconditionally archive the resulting object file in the libgit.a to be picked up by the linker. There are a few things to note in the shape of the code base after this change: - We no longer use NO_UNCOMPRESS2 knob; if the system header <zlib.h> claims a version that is more cent than the library actually is, this would break, but it is easy to add it back when we find such a system. - The object file compat/zlib-uncompress2.o is always compiled and archived in libgit.a, just like a few other compat/ object files already are. - The inclusion of <zlib.h> is done in <git-compat-util.h>; we used to do so from <cache.h> which includes <git-compat-util.h> as the first thing it does, so from the *.c codes, there is no practical change. - Until objects in libgit.a that is already used gains a reference to the function, the reftable code will be the only one that wants it, so libgit.a on the linker command line needs to appear once more at the end to satisify the mutual dependency. - Beat found a trick used by OpenSSL to avoid making the conditionally-compiled object truly empty (apparently because they had to deal with compilers that do not want to see an effectively empty input file). Our compat/zlib-uncompress2.c file borrows the same trick for portabilty. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: Beat Bolli <dev+git@drbeat.li> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-17wrapper: add a helper to generate numbers from a CSPRNGLibravatar brian m. carlson1-0/+19
There are many situations in which having access to a cryptographically secure pseudorandom number generator (CSPRNG) is helpful. In the future, we'll encounter one of these when dealing with temporary files. To make this possible, let's add a function which reads from a system CSPRNG and returns some bytes. We know that all systems will have such an interface. A CSPRNG is required for a secure TLS or SSH implementation and a Git implementation which provided neither would be of little practical use. In addition, POSIX is set to standardize getentropy(2) in the next version, so in the (potentially distant) future we can rely on that. For systems which lack one of the other interfaces, we provide the ability to use OpenSSL's CSPRNG. OpenSSL is highly portable and functions on practically every known OS, and we know it will have access to some source of cryptographically secure randomness. We also provide support for the arc4random in libbsd for folks who would prefer to use that. Because this is a security sensitive interface, we take some precautions. We either succeed by filling the buffer completely as we requested, or we fail. We don't return partial data because the caller will almost never find that to be a useful behavior. Specify a makefile knob which users can use to specify one or more suitable CSPRNGs, and turn the multiple string options into a set of defines, since we cannot match on strings in the preprocessor. We allow multiple options to make the job of handling this in autoconf easier. The order of options is important here. On systems with arc4random, which is most of the BSDs, we use that, since, except on MirBSD and macOS, it uses ChaCha20, which is extremely fast, and sits entirely in userspace, avoiding a system call. We then prefer getrandom over getentropy, because the former has been available longer on Linux, and then OpenSSL. Finally, if none of those are available, we use /dev/urandom, because most Unix-like operating systems provide that API. We prefer options that don't involve device files when possible because those work in some restricted environments where device files may not be available. Set the configuration variables appropriately for Linux and the BSDs, including macOS, as well as Windows and NonStop. We specifically only consider versions which receive publicly available security support here. For the same reason, we don't specify getrandom(2) on Linux, because CentOS 7 doesn't support it in glibc (although its kernel does) and we don't want to resort to making syscalls. Finally, add a test helper to allow this to be tested by hand and in tests. We don't add any tests, since invoking the CSPRNG is not likely to produce interesting, reproducible results. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-10Merge branch 'ab/usage-die-message'Libravatar Junio C Hamano1-1/+3
Code clean-up to hide vreportf() from public API. * ab/usage-die-message: config API: use get_error_routine(), not vreportf() usage.c + gc: add and use a die_message_errno() gc: return from cmd_gc(), don't call exit() usage.c API users: use die_message() for error() + exit 128 usage.c API users: use die_message() for "fatal :" + exit 128 usage.c: add a die_message() routine
2022-01-05Merge branch 'jc/flex-array-definition'Libravatar Junio C Hamano1-2/+11
The conditions to choose different definitions of the FLEX_ARRAY macro for vendor compilers has been simplified to make it easier to maintain. * jc/flex-array-definition: flex-array: simplify compiler-specific workaround
2021-12-10Merge branch 'cb/mingw-gmtime-r'Libravatar Junio C Hamano1-1/+3
Build fix on Windows. * cb/mingw-gmtime-r: mingw: avoid fallback for {local,gm}time_r()
2021-12-10Merge branch 'bc/require-c99'Libravatar Junio C Hamano1-0/+13
Weather balloon to break people with compilers that do not support C99. * bc/require-c99: git-compat-util: add a test balloon for C99 support
2021-12-08flex-array: simplify compiler-specific workaroundLibravatar Junio C Hamano1-2/+11
We use "type array[];" syntax for the flex-array member at the end of a struct under C99 or later, except when we are building with older SUNPRO_C compilers. As we find more vendor compilers that claim to grok C99 but not understand the flex-array syntax, the existing "If we are using C99, but not with these compilers..." conditional will keep growing. Make it more manageable by listing vendor-specific exceptions earlier, with the expectation that new exceptions will not be combined into existing ones to make the condition longer, and instead will be implemented as a new "#elif" in the cascade of similar to old SUNPRO_C, we can just add a single line #elif defined(_MSC_VER) immediately before "#elif defined(__GNUC__)" to cause us to fallback to the safer but a bit wasteful version. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-07config API: use get_error_routine(), not vreportf()Libravatar Ævar Arnfjörð Bjarmason1-1/+0
Change the git_die_config() function added in 5a80e97c827 (config: add `git_die_config()` to the config-set API, 2014-08-07) to use the public callbacks in the usage.[ch] API instead of the the underlying vreportf() function. In preceding commits the rest of the vreportf() users outside of usage.c was migrated to die_message(), so we can now make it "static". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-07usage.c + gc: add and use a die_message_errno()Libravatar Ævar Arnfjörð Bjarmason1-0/+1
Change the "error: " output when we exit with 128 due to gc.log errors to use a "fatal: " prefix instead. To do this add a die_message_errno() a sibling function to the die_errno() added in a preceding commit. Before this we'd expect report_last_gc_error() to return -1 from error_errno() in this case. It already treated a status of 0 and 1 specially. Let's just document that anything that's not 0 or 1 should be returned. We could also retain the "ret < 0" behavior here without hardcoding 128 by returning -128, and having the caller do a "return -ret", but I think this makes more sense, and preserves the path from die_message*()'s return value to the "return" without hardcoding "128". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-07usage.c: add a die_message() routineLibravatar Ævar Arnfjörð Bjarmason1-0/+2
We have code in various places that would like to call die(), but wants to defer the exit(128) it would invoke, e.g. to print an additional message, or adjust the exit code. Add a die_message() helper routine to bridge this gap in the API. Functionally this behaves just like the error() routine, except it'll print a "fatal: " prefix, and it will return with 128 instead of -1, this is so that caller can pass the return value to "exit()", instead of having to hardcode "exit(128)". Note that as with the other routines the "die_message_builtin" needs to return "void" and otherwise conform to the "report_fn" signature. As we'll see in a subsequent commit callers will want to replace e.g. their default "die_routine" with a "die_message_routine". For now we're just adding the routine and making die_builtin() in usage.c itself use it. In order to do that we need to add a get_die_message_routine() function, which works like the other get_*_routine() functions in usage.c. There is no set_die_message_rotine(), as it hasn't been needed yet. We can add it if we ever need it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-01git-compat-util: add a test balloon for C99 supportLibravatar brian m. carlson1-0/+13
The C99 standard was released in January 1999, now 22 years ago. It provides a variety of useful features, including variadic arguments for macros, declarations after statements, designated initializers, and a wide variety of other useful features, many of which we already use. We'd like to take advantage of these features, but we want to be cautious. As far as we know, all major compilers now support C99 or a later C standard, such as C11 or C17. POSIX has required C99 support as a requirement for the 2001 revision, so we can safely assume any POSIX system which we are interested in supporting has C99. Even MSVC, long a holdout against modern C, now supports both C11 and C17 with an appropriate update. Moreover, even if people are using an older version of MSVC on these systems, they will generally need some implementation of the standard Unix utilities for the testsuite, and GNU coreutils, the most common option, has required C99 since 2009. Therefore, we can safely assume that a suitable version of GCC or clang is available to users even if their version of MSVC is not sufficiently capable. Let's add a test balloon to git-compat-util.h to see if anyone is using an older compiler. We'll add a comment telling people how to enable this functionality on GCC and Clang, even though modern versions of both will automatically do the right thing, and ask people still experiencing a problem to report that to us on the list. Note that C89 compilers don't provide the __STDC_VERSION__ macro, so we use a well-known hack of using "- 0". On compilers with this macro, it doesn't change the value, and on C89 compilers, the macro will be replaced with nothing, and our value will be 0. For sparse, we explicitly request the gnu99 style because we've traditionally taken advantage of some GCC- and clang-specific extensions when available and we'd like to retain the ability to do that. sparse also defaults to C89 without it, so things will fail for us if we don't. Update the cmake configuration to require C11 for MSVC. We do this because this will make MSVC to use C11, since it does not explicitly support C99. We do this with a compiler options because setting the C_STANDARD option does not work in our CI on MSVC and at the moment, we don't want to require C11 for Unix compilers. In the Makefile, don't set any compiler flags for the compiler itself, since on some systems, such as FreeBSD, we actually need C11, and asking for C99 causes things to fail to compile. The error message should make it obvious what's going wrong and allow a user to set the appropriate option when building in the event they're using a Unix compiler that doesn't support it by default. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-29Merge branch 'mc/clean-smudge-with-llp64'Libravatar Junio C Hamano1-0/+25
The clean/smudge conversion code path has been prepared to better work on platforms where ulong is narrower than size_t. * mc/clean-smudge-with-llp64: clean/smudge: allow clean filters to process extremely large files odb: guard against data loss checking out a huge file git-compat-util: introduce more size_t helpers odb: teach read_blob_entry to use size_t t1051: introduce a smudge filter test for extremely large files test-lib: add prerequisite for 64-bit platforms test-tool genzeros: generate large amounts of data more efficiently test-genzeros: allow more than 2G zeros in Windows
2021-11-29Merge branch 'jc/unsetenv-returns-an-int'Libravatar Junio C Hamano1-1/+1
The compatibility implementation for unsetenv(3) were written to mimic ancient, non-POSIX, variant seen in an old glibc; it has been changed to return an integer to match the more modern era. * jc/unsetenv-returns-an-int: unsetenv(3) returns int, not void
2021-11-27mingw: avoid fallback for {local,gm}time_r()Libravatar Carlo Marcelo Arenas Belón1-1/+3
mingw-w64's pthread_unistd.h had a bug that mistakenly (because there is no support for the *lockfile() functions required[1]) defined _POSIX_THREAD_SAFE_FUNCTIONS and that was being worked around since 3ecd153a3b (compat/mingw: support MSys2-based MinGW build, 2016-01-14). The bug was fixed in winphtreads, but as a side effect, leaves the reentrant functions from time.h no longer visible and therefore breaks the build. Since the intention all along was to avoid using the fallback functions, formalize the use of POSIX by setting the corresponding feature flag and compile out the implementation for the fallback functions. [1] https://unix.org/whitepapers/reentrant.html Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-03git-compat-util: introduce more size_t helpersLibravatar Johannes Schindelin1-0/+25
We will use them in the next commit. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29unsetenv(3) returns int, not voidLibravatar Junio C Hamano1-1/+1
This compatilibity implementation has been returning a wrong type, ever since 731043fd (Add compat/unsetenv.c ., 2006-01-25) added to the system, yet nobody noticed it in the past 16 years, presumably because no code checks failures in their unsetenv() calls. Sigh. For now, make it always succeed. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29wrapper: remove xunsetenv()Libravatar Carlo Marcelo Arenas Belón1-1/+0
Remove the unused wrapper function. Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-06Merge branch 'ab/repo-settings-cleanup'Libravatar Junio C Hamano1-0/+2
Code cleanup. * ab/repo-settings-cleanup: repository.h: don't use a mix of int and bitfields repo-settings.c: simplify the setup read-cache & fetch-negotiator: check "enum" values in switch() environment.c: remove test-specific "ignore_untracked..." variable wrapper.c: add x{un,}setenv(), and use xsetenv() in environment.c
2021-09-23Merge branch 'cb/unix-sockets-with-windows'Libravatar Junio C Hamano1-0/+3
Adjust credential-cache helper to Windows. * cb/unix-sockets-with-windows: git-compat-util: include declaration for unix sockets in windows credential-cache: check for windows specific errors t0301: fixes for windows compatibility
2021-09-22wrapper.c: add x{un,}setenv(), and use xsetenv() in environment.cLibravatar Ævar Arnfjörð Bjarmason1-0/+2
Add fatal wrappers for setenv() and unsetenv(). In d7ac12b25d3 (Add set_git_dir() function, 2007-08-01) we started checking its return value, and since 48988c4d0c3 (set_git_dir: die when setenv() fails, 2018-03-30) we've had set_git_dir_1() die if we couldn't set it. Let's provide a wrapper for both, this will be useful in many other places, a subsequent patch will make another use of xsetenv(). The checking of the return value here is over-eager according to setenv(3) and POSIX. It's documented as returning just -1 or 0, so perhaps we should be checking -1 explicitly. Let's just instead die on any non-zero, if our C library is so broken as to return something else than -1 on error (and perhaps not set errno?) the worst we'll do is die with a nonsensical errno value, but we'll want to die in either case. Let's make these return "void" instead of "int". As far as I can tell there's no other x*() wrappers that needed to make the decision of deviating from the signature in the C library, but since their return value is only used to indicate errors (so we'd die here), we can catch unreachable code such as if (xsetenv(...) < 0) [...]; I think it would be OK skip the NULL check of the "name" here for the calls to die_errno(). Almost all of our setenv() callers are taking a constant string hardcoded in the source as the first argument, and for the rest we can probably assume they've done the NULL check themselves. Even if they didn't, modern C libraries are forgiving about it (e.g. glibc formatting it as "(null)"), on those that aren't, well, we were about to die anyway. But let's include the check anyway for good measure. 1. https://pubs.opengroup.org/onlinepubs/009604499/functions/setenv.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-14git-compat-util: include declaration for unix sockets in windowsLibravatar Carlo Marcelo Arenas Belón1-0/+3
Available since Windows 10 release 1803 and Windows Server 2019. NO_UNIX_SOCKETS is still the default for Windows builds, as they need to keep backward compatibility with releases up to Windows 7, but allow including the header otherwise. Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-03gettext: remove optional non-standard parens in N_() definitionLibravatar Ævar Arnfjörð Bjarmason1-4/+0
Remove the USE_PARENS_AROUND_GETTEXT_N compile-time option which was meant to catch an inadvertent mistake which is too obscure to maintain this facility. The backstory of how USE_PARENS_AROUND_GETTEXT_N came about is: When I added the N_() macro in 65784830366 (i18n: add no-op _() and N_() wrappers, 2011-02-22) it was defined as: #define N_(msgid) (msgid) This is non-standard C, as was noticed and fixed in 642f85faab2 (i18n: avoid parenthesized string as array initializer, 2011-04-07). I.e. this needed to be defined as: #define N_(msgid) msgid Then in e62cd35a3e8 (i18n: log: mark parseopt strings for translation, 2012-08-20) when "builtin_log_usage" was marked for translation the string concatenation for passing to usage() added in 1c370ea4e51 (Show usage string for 'git log -h', 'git show -h' and 'git diff -h', 2009-08-06) was faithfully preserved: - "git log [<options>] [<since>..<until>] [[--] <path>...]\n" - " or: git show [options] <object>...", + N_("git log [<options>] [<since>..<until>] [[--] <path>...]\n") + N_(" or: git show [options] <object>..."), This was then fixed to be the expected array of usage strings in e66dc0cc4b1 (log.c: fix translation markings, 2015-01-06) rather than a string with multiple "\n"-delimited usage strings, and finally in 290c8e7a3fe (gettext.h: add parentheses around N_ expansion if supported, 2015-01-11) USE_PARENS_AROUND_GETTEXT_N was added to ensure this mistake didn't happen again. I think that even if this was a N_()-specific issue this USE_PARENS_AROUND_GETTEXT_N facility wouldn't be worth it, the issue would be too rare to worry about. But I also think that 290c8e7a3fe which introduced USE_PARENS_AROUND_GETTEXT_N misattributed the problem. The issue wasn't with the N_() macro added in e62cd35a3e8, but that before the N_() macro existed in the codebase the initial migration to parse_options() in 1c370ea4e51 continued passsing in a "\n"-delimited string, when the new API it was migrating to supported and expected the passing of an array. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-16Merge branch 'ew/mmap-failures'Libravatar Junio C Hamano1-0/+1
Error message update. * ew/mmap-failures: xmmap: inform Linux users of tuning knobs on ENOMEM
2021-07-08Merge branch 'ar/typofix'Libravatar Junio C Hamano1-1/+1
Typofixes. * ar/typofix: *: fix typos which duplicate a word
2021-06-29xmmap: inform Linux users of tuning knobs on ENOMEMLibravatar Eric Wong1-0/+1
Linux users may benefit from additional information on how to avoid ENOMEM from mmap despite the system having enough RAM to accomodate them. We can't reliably unmap pack windows to work around the issue since malloc and other library routines may mmap without our knowledge. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-14*: fix typos which duplicate a wordLibravatar Andrei Rybak1-1/+1
Fix typos in documentation, code comments, and RelNotes which repeat various words. In trivial cases, just delete the duplicated word and rewrap text, if needed. Reword the affected sentence in Documentation/RelNotes/1.8.4.txt for it to make sense. Signed-off-by: Andrei Rybak <rybak.a.v@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-10Merge branch 'jn/size-t-casted-to-off-t-fix'Libravatar Junio C Hamano1-4/+2
Rewrite code that triggers undefined behaiour warning. * jn/size-t-casted-to-off-t-fix: xsize_t: avoid implementation defined behavior when len < 0
2021-05-19xsize_t: avoid implementation defined behavior when len < 0Libravatar Jonathan Nieder1-4/+2
The xsize_t helper aims to safely convert an off_t to a size_t, erroring out when a file offset is too large to fit into a memory address. It does this by using two casts: size_t size = (size_t) len; if (len != (off_t) size) ... error out ... On a platform with sizeof(size_t) < sizeof(off_t), this check is safe and correct. The first cast truncates to a size_t by finding the remainder modulo SIZE_MAX+1 (see C99 section 6.3.1.3 Signed and unsigned integers) and the second promotes to an off_t, meaning the result is true if and only if len is representable as a size_t. On other platforms, this two-casts strategy still works well (always succeeds) for len >= 0. But for len < 0, when the first cast succeeds and produces SIZE_MAX + 1 + len, the resulting value is too large to be represented as an off_t, so the second cast produces implementation defined behavior. In practice, it is likely to produce a result of true despite len not being representable as size_t. Simplify by replacing with a more straightforward check: compare len to the relevant bounds and then cast it. (To avoid a -Wsign-compare warning, after checking that len >= 0, we explicitly convert to a sufficiently-large unsigned type before comparing to SIZE_MAX.) In practice, this is not likely to come up since typical callers use nonnegative len. Still, it's helpful to handle this case to make the behavior easy to reason about. Historical note: the original bounds-checking in 46be82dfd0 (xsize_t: check whether we lose bits, 2010-07-28) did not produce this implementation-defined behavior, though it still did not handle negative offsets. It was not until 73560c793a (git-compat-util.h: xsize_t() - avoid -Wsign-compare warnings, 2017-09-21) introduced the double cast that the implementation-defined behavior was triggered. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-13Merge branch 'tb/precompose-prefix-simplify'Libravatar Junio C Hamano1-0/+5
Streamline the codepath to fix the UTF-8 encoding issues in the argv[] and the prefix on macOS. * tb/precompose-prefix-simplify: macOS: precompose startup_info->prefix precompose_utf8: make precompose_string_if_needed() public
2021-04-05precompose_utf8: make precompose_string_if_needed() publicLibravatar Torsten Bögershausen1-0/+5
commit 5c327502 (MacOS: precompose_argv_prefix(), 2021-02-03) uses the function precompose_string_if_needed() internally. It is only used from precompose_argv_prefix() and therefore static in compat/precompose_utf8.c Expose this function, it will be used in the next commit. While there, allow passing a NULL pointer, which will return NULL. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-22Merge branch 'jk/open-dotgitx-with-nofollow'Libravatar Junio C Hamano1-0/+7
It does not make sense to make ".gitattributes", ".gitignore" and ".mailmap" symlinks, as they are supposed to be usable from the object store (think: bare repositories where HEAD:.mailmap etc. are used). When these files are symbolic links, we used to read the contents of the files pointed by them by mistake, which has been corrected. * jk/open-dotgitx-with-nofollow: mailmap: do not respect symlinks for in-tree .mailmap exclude: do not respect symlinks for in-tree .gitignore attr: do not respect symlinks for in-tree .gitattributes exclude: add flags parameter to add_patterns() attr: convert "macro_ok" into a flags field add open_nofollow() helper
2021-03-13git-compat-util.h: drop trailing semicolon from macro definitionLibravatar René Scharfe1-1/+1
Make CALLOC_ARRAY usable like a function by requiring callers to supply the trailing semicolon, which all of the current ones already do. With the extra semicolon e.g. the following code wouldn't compile because it disconnects the "else" from the "if": if (condition) CALLOC_ARRAY(ptr, n); else whatever(); Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-08Sync with Git 2.30.2 for CVE-2021-21300Libravatar Junio C Hamano1-0/+5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-04Merge branch 'jk/open-returns-eintr'Libravatar Junio C Hamano1-0/+6
Work around platforms whose open() is reported to return EINTR (it shouldn't, as we do our signals with SA_RESTART). * jk/open-returns-eintr: config.mak.uname: enable OPEN_RETURNS_EINTR for macOS Big Sur Makefile: add OPEN_RETURNS_EINTR knob
2021-02-26Makefile: add OPEN_RETURNS_EINTR knobLibravatar Jeff King1-0/+6
On some platforms, open() reportedly returns EINTR when opening regular files and we receive a signal (usually SIGALRM from our progress meter). This shouldn't happen, as open() should be a restartable syscall, and we specify SA_RESTART when setting up the alarm handler. So it may actually be a kernel or libc bug for this to happen. But it has been reported on at least one version of Linux (on a network filesystem): https://lore.kernel.org/git/c8061cce-71e4-17bd-a56a-a5fed93804da@neanderfunk.de/ as well as on macOS starting with Big Sur even on a regular filesystem. We can work around it by retrying open() calls that get EINTR, just as we do for read(), etc. Since we don't ever _want_ to interrupt an open() call, we can get away with just redefining open, rather than insisting all callsites use xopen(). We actually do have an xopen() wrapper already (and it even does this retry, though there's no indication of it being an observed problem back then; it seems simply to have been lifted from xread(), etc). But it is used hardly anywhere, and isn't suitable for general use because it will die() on error. In theory we could combine the two, but it's awkward to do so because of the variable-args interface of open(). This patch adds a Makefile knob for enabling the workaround. It's not enabled by default for any platforms in config.mak.uname yet, as we don't have enough data to decide how common this is (I have not been able to reproduce on either Linux or Big Sur myself). It may be worth enabling preemptively anyway, since the cost is pretty low (if we don't see an EINTR, it's just an extra conditional). However, note that we must not enable this on Windows. It doesn't do anything there, and the macro overrides the existing mingw_open() redirection. I've added a preemptive #undef here in the mingw header (which is processed first) to just quietly disable it (we could also make it an #error, but there is little point in being so aggressive). Reported-by: Aleksey Kliger <alklig@microsoft.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-16add open_nofollow() helperLibravatar Jeff King1-0/+7
Some callers of open() would like to use O_NOFOLLOW, but it is not available on all platforms. Let's abstract this into a helper function so we can provide system-specific implementations. Some light web-searching reveals that we might be able to get something similar on Windows using FILE_FLAG_OPEN_REPARSE_POINT. I didn't dig into this further. For other systems without O_NOFOLLOW or any equivalent, we have two options for fallback: - we can just open anyway, following symlinks; this may have security implications (e.g., following untrusted in-tree symlinks) - we can determine whether the path is a symlink with lstat(). This is slower (two syscalls instead of one), but that may be acceptable for infrequent uses like looking up .gitattributes files (especially because we can get away with a single syscall for the common case of ENOENT). It's also racy, but should be sufficient for our needs (we are worried about in-tree symlinks that we ourselves would have previously created). We could make it non-racy at the cost of making it even slower, by doing an fstat() on the opened descriptor and comparing the dev/ino fields to the original lstat(). This patch implements the lstat() option in its slightly-faster racy form. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-12Merge branch 'tb/precompose-prefix-too'Libravatar Junio C Hamano1-2/+2
When commands are started from a subdirectory, they may have to compare the path to the subdirectory (called prefix and found out from $(pwd)) with the tracked paths. On macOS, $(pwd) and readdir() yield decomposed path, while the tracked paths are usually normalized to the precomposed form, causing mismatch. This has been fixed by taking the same approach used to normalize the command line arguments. * tb/precompose-prefix-too: MacOS: precompose_argv_prefix()
2021-02-12Sync with 2.29.3Libravatar Johannes Schindelin1-0/+5
* maint-2.29: Git 2.29.3 Git 2.28.1 Git 2.27.1 Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.28.1Libravatar Johannes Schindelin1-0/+5
* maint-2.28: Git 2.28.1 Git 2.27.1 Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.27.1Libravatar Johannes Schindelin1-0/+5
* maint-2.27: Git 2.27.1 Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.26.3Libravatar Johannes Schindelin1-0/+5
* maint-2.26: Git 2.26.3 Git 2.25.5 Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.24.4Libravatar Johannes Schindelin1-0/+5
* maint-2.24: Git 2.24.4 Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.23.4Libravatar Johannes Schindelin1-0/+5
* maint-2.23: Git 2.23.4 Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.22.5Libravatar Johannes Schindelin1-0/+5
* maint-2.22: Git 2.22.5 Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.21.4Libravatar Johannes Schindelin1-0/+5
* maint-2.21: Git 2.21.4 Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path
2021-02-12Sync with 2.20.5Libravatar Johannes Schindelin1-0/+5
* maint-2.20: Git 2.20.5 Git 2.19.6 Git 2.18.5 Git 2.17.6 unpack_trees(): start with a fresh lstat cache run-command: invalidate lstat cache after a command finished checkout: fix bug that makes checkout follow symlinks in leading path