summaryrefslogtreecommitdiff
path: root/git-compat-util.h
AgeCommit message (Collapse)AuthorFilesLines
2019-10-15Merge branch 'ew/hashmap'Libravatar Junio C Hamano1-0/+38
Code clean-up of the hashmap API, both users and implementation. * ew/hashmap: hashmap_entry: remove first member requirement from docs hashmap: remove type arg from hashmap_{get,put,remove}_entry OFFSETOF_VAR macro to simplify hashmap iterators hashmap: introduce hashmap_free_entries hashmap: hashmap_{put,remove} return hashmap_entry * hashmap: use *_entry APIs for iteration hashmap_cmp_fn takes hashmap_entry params hashmap_get{,_from_hash} return "struct hashmap_entry *" hashmap: use *_entry APIs to wrap container_of hashmap_get_next returns "struct hashmap_entry *" introduce container_of macro hashmap_put takes "struct hashmap_entry *" hashmap_remove takes "const struct hashmap_entry *" hashmap_get takes "const struct hashmap_entry *" hashmap_add takes "struct hashmap_entry *" hashmap_get_next takes "const struct hashmap_entry *" hashmap_entry_init takes "struct hashmap_entry *" packfile: use hashmap_entry in delta_base_cache_entry coccicheck: detect hashmap_entry.hash assignment diff: use hashmap_entry_init on moved_entry.ent
2019-10-09Merge branch 'js/diff-rename-force-stable-sort'Libravatar Junio C Hamano1-3/+6
The rename detection logic sorts a list of rename source candidates by similarity to pick the best candidate, which means that a tie between sources with the same similarity is broken by the original location in the original candidate list (which is sorted by path). Force the sorting by similarity done with a stable sort, which is not promised by system supplied qsort(3), to ensure consistent results across platforms. * js/diff-rename-force-stable-sort: diffcore_rename(): use a stable sort Move git_sort(), a stable sort, into into libgit.a
2019-10-09Merge branch 'sg/name-rev-cutoff-underflow-fix'Libravatar Junio C Hamano1-0/+1
Integer arithmetic fix. * sg/name-rev-cutoff-underflow-fix: name-rev: avoid cutoff timestamp underflow
2019-10-07OFFSETOF_VAR macro to simplify hashmap iteratorsLibravatar Eric Wong1-0/+13
While we cannot rely on a `__typeof__' operator being portable to use with `offsetof'; we can calculate the pointer offset using an existing pointer and the address of a member using pointer arithmetic for compilers without `__typeof__'. This allows us to simplify usage of hashmap iterator macros by not having to specify a type when a pointer of that type is already given. In the future, list iterator macros (e.g. list_for_each_entry) may also be implemented using OFFSETOF_VAR to save hackers the trouble of using container_of/list_entry macros and without relying on non-portable `__typeof__'. v3: use `__typeof__' to avoid clang warnings Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07hashmap: use *_entry APIs to wrap container_ofLibravatar Eric Wong1-0/+15
Using `container_of' can be verbose and choosing names for intermediate "struct hashmap_entry" pointers is a hard problem. So introduce "*_entry" APIs inspired by similar linked-list APIs in the Linux kernel. Unfortunately, `__typeof__' is not portable C, so we need an extra parameter to specify the type. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07introduce container_of macroLibravatar Eric Wong1-0/+10
This macro is popular within the Linux kernel for supporting intrusive data structures such as linked lists, red-black trees, and chained hash tables while allowing the compiler to do type checking. Later patches will use container_of() to remove the limitation of "hashmap_entry" being location-dependent. This will complete the transition to compile-time type checking for the hashmap API. This macro already exists in our source as "list_entry" in list.h and making "list_entry" an alias to "container_of" as the Linux kernel has done is a possibility. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-02Move git_sort(), a stable sort, into into libgit.aLibravatar Johannes Schindelin1-3/+6
The `qsort()` function is not guaranteed to be stable, i.e. it does not promise to maintain the order of items it is told to consider equal. In contrast, the `git_sort()` function we carry in `compat/qsort.c` _is_ stable, by virtue of implementing a merge sort algorithm. In preparation for using a stable sort in Git's rename detection, move the stable sort into `libgit.a` so that it is compiled in unconditionally, and rename it to `git_stable_qsort()`. Note: this also makes the hack obsolete that was introduced in fe21c6b285d (mingw: reencode environment variables on the fly (UTF-16 <-> UTF-8), 2018-10-30), where we included `compat/qsort.c` directly in `compat/mingw.c` to use the stable sort. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-28name-rev: avoid cutoff timestamp underflowLibravatar SZEDER Gábor1-0/+1
When 'git name-rev' is invoked with commit-ish parameters, it tries to save some work, and doesn't visit commits older than the committer date of the oldest given commit minus a one day worth of slop. Since our 'timestamp_t' is an unsigned type, this leads to a timestamp underflow when the committer date of the oldest given commit is within a day of the UNIX epoch. As a result the cutoff timestamp ends up far-far in the future, and 'git name-rev' doesn't visit any commits, and names each given commit as 'undefined'. Check whether subtracting the slop from the oldest committer date would lead to an underflow, and use no cutoff in that case. We don't have a TIME_MIN constant, dddbad728c (timestamp_t: a new data type for timestamps, 2017-04-26) didn't add one, so do it now. Note that the type of the cutoff timestamp variable used to be signed before 5589e87fd8 (name-rev: change a "long" variable to timestamp_t, 2017-05-20). The behavior was still the same even back then, but the underflow didn't happen when substracting the slop from the oldest committer date, but when comparing the signed cutoff timestamp with unsigned committer dates in name_rev(). IOW, this underflow bug is as old as 'git name-rev' itself. Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-13packfile: drop release_pack_memory()Libravatar Jeff King1-3/+0
Long ago, in 97bfeb34df (Release pack windows before reporting out of memory., 2006-12-24), we taught xmalloc() and friends to try unmapping pack windows when malloc() failed. It's unlikely that his helps a lot in practice, and it has some downsides. First, the downsides: 1. It makes xmalloc() not thread-safe. We've worked around this in pack-objects.c, which installs its own locking version of the try_to_free_routine(). But other threaded code doesn't. 2. It makes the system as a whole harder to reason about. Functions which allocate heap memory under the hood may have farther-reaching effects than expected. That might be worth the tradeoff if there's a benefit. But in practice, it seems unlikely. We're generally dealing with mmap'd files, so the OS is going to do a much better job at responding to memory pressure by dropping individual pages (the exception is systems with NO_MMAP, but even there the OS can probably respond just as well with swapping). So the only thing we're really freeing is address space. On 64-bit systems, we have plenty of that to go around. On 32-bit systems, it could possibly help. But around the same time we made two other changes: 77ccc5bbd1 (Introduce new config option for mmap limit., 2006-12-23) and 60bb8b1453 (Fully activate the sliding window pack access., 2006-12-23). Together that means that a 32-bit system should have no more than 256MB total of packed-git mmaps at one time, split between a few 32MB windows. It's unlikely we have any address space problems since then, but we don't have any data since the features were all added at the same time. Likewise, xmmap() will try to free memory. At first glance, it seems like we'd need this (when we try to mmap a new window, we might need to close an old one to save address space on a 32-bit system). But we're saved again by core.packedGitLimit: if we're going to exceed our 256MB limit, we'll close an existing window before we even call mmap(). So it seems unlikely that this feature is actually doing anything useful. And while we don't have reports of it harming anything (probably because it rarely if ever kicks in), it would be nice to simplify the system overall. This patch drops the whole try_to_free system from xmalloc(), as well as the manual pack memory release in xmmap(). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25msvc: add a compile-time flag to allow detailed heap debuggingLibravatar Jeff Hostetler1-0/+9
MS Visual C comes with a few neat features we can use to analyze the heap consumption (i.e. leaks, max memory, etc). With this patch, we introduce support via the build-time flag `USE_MSVC_CRTDBG`. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-13Merge branch 'cc/access-on-aix-workaround'Libravatar Junio C Hamano1-1/+11
Workaround for standard-compliant but less-than-useful behaviour of access(2) for the root user. * cc/access-on-aix-workaround: git-compat-util: work around for access(X_OK) under root
2019-05-13Merge branch 'dl/no-extern-in-func-decl'Libravatar Junio C Hamano1-66/+66
Mechanically and systematically drop "extern" from function declarlation. * dl/no-extern-in-func-decl: *.[ch]: manually align parameter lists *.[ch]: remove extern from function declarations using sed *.[ch]: remove extern from function declarations using spatch
2019-05-09Merge branch 'ss/msvc-path-utils-fix'Libravatar Junio C Hamano1-0/+1
An earlier update for MinGW and Cygwin accidentally broke MSVC build, which has been fixed. * ss/msvc-path-utils-fix: MSVC: include compat/win32/path-utils.h for MSVC, too, for real_path()
2019-05-05*.[ch]: manually align parameter listsLibravatar Denton Liu1-2/+2
In previous patches, extern was mechanically removed from function declarations without care to formatting, causing parameter lists to be misaligned. Manually format changed sections such that the parameter lists should be realigned. Viewing this patch with 'git diff -w' should produce no output. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-05*.[ch]: remove extern from function declarations using sedLibravatar Denton Liu1-10/+10
There has been a push to remove extern from function declarations. Finish the job by removing all instances of "extern" for function declarations in headers using sed. This was done by running the following on my system with sed 4.2.2: $ git ls-files \*.{c,h} | grep -v ^compat/ | xargs sed -i'' -e 's/^\(\s*\)extern \([^(]*([^*]\)/\1\2/' Files under `compat/` are intentionally excluded as some are directly copied from external sources and we should avoid churning them as much as possible. Then, leftover instances of extern were found by running $ git grep -w -C3 extern \*.{c,h} and manually checking the output. No other instances were found. Note that the regex used specifically excludes function variables which _should_ be left as extern. Not the most elegant way to do it but it gets the job done. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-05*.[ch]: remove extern from function declarations using spatchLibravatar Denton Liu1-54/+54
There has been a push to remove extern from function declarations. Remove some instances of "extern" for function declarations which are caught by Coccinelle. Note that Coccinelle has some difficulty with processing functions with `__attribute__` or varargs so some `extern` declarations are left behind to be dealt with in a future patch. This was the Coccinelle patch used: @@ type T; identifier f; @@ - extern T f(...); and it was run with: $ git ls-files \*.{c,h} | grep -v ^compat/ | xargs spatch --sp-file contrib/coccinelle/noextern.cocci --in-place Files under `compat/` are intentionally excluded as some are directly copied from external sources and we should avoid churning them as much as possible. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25git-compat-util: work around for access(X_OK) under rootLibravatar Clément Chigot1-1/+11
On AIX, access(X_OK) may succeed when run as root even if the execution isn't possible. This behavior is allowed by POSIX which says: ... for a process with appropriate privileges, an implementation may indicate success for X_OK even if execute permission is not granted to any user. It can lead hook programs to have their execution refused: git commit -m content fatal: cannot exec '.git/hooks/pre-commit': Permission denied Add NEED_ACCESS_ROOT_HANDLER in order to use an access helper function. It checks with stat if any executable flags is set when the current user is root. Signed-off-by: Clément Chigot <clement.chigot@atos.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-09MSVC: include compat/win32/path-utils.h for MSVC, too, for real_path()Libravatar Sven Strickroth1-0/+1
A path such as 'c:/somepath/submodule/../.git/modules/submodule' wasn't resolved correctly any more, because the *nix variant of offset_1st_component is used instead of the Win32 specific version. Regression was introduced in commit 1cadad6f6 when mingw_offset_1st_component was moved from mingw.c which is included by msvc.c to a separate file. Then, the new file "compat/win32/path-utils.h" was only included for the __CYGWIN__ and __MINGW32__ cases in git-compat-util.h, the case for _MSC_VER was missing. Signed-off-by: Sven Strickroth <email@cs-ware.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22trace2: create new combined trace facilityLibravatar Jeff Hostetler1-0/+7
Create a new unified tracing facility for git. The eventual intent is to replace the current trace_printf* and trace_performance* routines with a unified set of git_trace2* routines. In addition to the usual printf-style API, trace2 provides higer-level event verbs with fixed-fields allowing structured data to be written. This makes post-processing and analysis easier for external tools. Trace2 defines 3 output targets. These are set using the environment variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT". These may be set to "1" or to an absolute pathname (just like the current GIT_TRACE). * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command summary data. * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE. It extends the output with columns for the command process, thread, repo, absolute and relative elapsed times. It reports events for child process start/stop, thread start/stop, and per-thread function nesting. * GIT_TR2_EVENT is a new structured format. It writes event data as a series of JSON records. Calls to trace2 functions log to any of the 3 output targets enabled without the need to call different trace_printf* or trace_performance* routines. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13Merge branch 'nd/fileno-may-be-macro'Libravatar Junio C Hamano1-0/+8
* nd/fileno-may-be-macro: git-compat-util: work around fileno(fp) that is a macro
2019-02-12git-compat-util: work around fileno(fp) that is a macroLibravatar Duy Nguyen1-0/+8
On various BSD's, fileno(fp) is implemented as a macro that directly accesses the fields in the FILE * object, which breaks a function that accepts a "void *fp" parameter and calls fileno(fp) and expect it to work. Work it around by adding a compile-time knob FILENO_IS_A_MACRO that inserts a real helper function in the middle of the callchain. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18Merge branch 'rb/hpe'Libravatar Junio C Hamano1-0/+13
Portability updates for the HPE NonStop platform. * rb/hpe: compat/regex/regcomp.c: define intptr_t and uintptr_t on NonStop git-compat-util.h: add FLOSS headers for HPE NonStop config.mak.uname: support for modern HPE NonStop config. transport-helper: drop read/write errno checks transport-helper: use xread instead of read
2019-01-14Merge branch 'nd/indentation-fix'Libravatar Junio C Hamano1-1/+1
Code cleanup. * nd/indentation-fix: Indent code with TABs
2019-01-14Merge branch 'tb/use-common-win32-pathfuncs-on-cygwin'Libravatar Junio C Hamano1-1/+2
Cygwin update. * tb/use-common-win32-pathfuncs-on-cygwin: git clone <url> C:\cygwin\home\USER\repo' is working (again)
2019-01-03git-compat-util.h: add FLOSS headers for HPE NonStopLibravatar Randall S. Becker1-0/+13
The HPE NonStop (a.k.a. __TANDEM) platform cannot build git without using the FLOSS package supplied by HPE. The convenient location for including the relevant headers is in this file. The NSIG define is also not defined on __TANDEM, so we define it here as 100 if it is not defined only for __TANDEM builds. Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-26git clone <url> C:\cygwin\home\USER\repo' is working (again)Libravatar Torsten Bögershausen1-1/+2
A regression for cygwin users was introduced with commit 05b458c, "real_path: resolve symlinks by hand". In the the commit message we read: The current implementation of real_path uses chdir() in order to resolve symlinks. Unfortunately this isn't thread-safe as chdir() affects a process as a whole... The old (and non-thread-save) OS calls chdir()/pwd() had been replaced by a string operation. The cygwin layer "knows" that "C:\cygwin" is an absolute path, but the new string operation does not. "git clone <url> C:\cygwin\home\USER\repo" fails like this: fatal: Invalid path '/home/USER/repo/C:\cygwin\home\USER\repo' The solution is to implement has_dos_drive_prefix(), skip_dos_drive_prefix() is_dir_sep(), offset_1st_component() and convert_slashes() for cygwin in the same way as it is done in 'Git for Windows' in compat/mingw.[ch] Extract the needed code into compat/win32/path-utils.[ch] and use it for cygwin as well. Reported-by: Steven Penny <svnpenn@gmail.com> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-09Indent code with TABsLibravatar Nguyễn Thái Ngọc Duy1-1/+1
We indent with TABs and sometimes for fine alignment, TABs followed by spaces, but never all spaces (unless the indentation is less than 8 columns). Indenting with spaces slips through in some places. Fix them. Imported code and compat/ are left alone on purpose. The former should remain as close as upstream as possible. The latter pretty much has separate maintainers, it's up to them to decide. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-21Merge branch 'cc/delta-islands'Libravatar Junio C Hamano1-0/+1
A few issues in the implementation of "delta-islands" feature has been corrected. * cc/delta-islands: pack-objects: fix off-by-one in delta-island tree-depth computation pack-objects: zero-initialize tree_depth/layer arrays pack-objects: fix tree_depth and layer invariants
2018-11-21pack-objects: zero-initialize tree_depth/layer arraysLibravatar Jeff King1-0/+1
Commit 108f530385 (pack-objects: move tree_depth into 'struct packing_data', 2018-08-16) started maintaining a tree_depth array that matches the "objects" array. We extend the array when: 1. The objects array is extended, in which case we use realloc to extend the tree_depth array. 2. A caller asks to store a tree_depth for object N, and this is the first such request; we create the array from scratch and store the value for N. In the latter case, though, we use regular xmalloc(), and the depth values for any objects besides N is undefined. This happens to not trigger a bug with the current code, but the reasons are quite subtle: - we never ask about the depth for any object with index i < N. This is because we store the depth immediately for all trees and blobs. So any such "i" must be a non-tree, and therefore we will never need to care about its depth (in fact, we really only care about the depth of trees). - there are no objects at this point with index i > N, because we always fill in the depth for a tree immediately after its object entry is created (we may still allocate uninitialized depth entries, but they'll be initialized by packlist_alloc() when it initializes the entry in the "objects" array). So it works, but only by chance. To be defensive, let's zero the array, which matches the "unset" values which would be handed out by oe_tree_depth() already. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14git-compat-util: prefer poll.h to sys/poll.hLibravatar Đoàn Trần Công Danh1-1/+4
POSIX specifies that <poll.h> is the correct header for poll(2) whereas <sys/poll.h> is only needed for some old libc. Let's follow the POSIX way by default. This effectively eliminates musl's warning: warning redirecting incorrect #include <sys/poll.h> to <poll.h> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31config: allow for platform-specific core.* config settingsLibravatar Johannes Schindelin1-0/+8
In the Git for Windows project, we have ample precendent for config settings that apply to Windows, and to Windows only. Let's formalize this concept by introducing a platform_core_config() function that can be #define'd in a platform-specific manner. This will allow us to contain platform-specific code better, as the corresponding variables no longer need to be exported so that they can be defined in environment.c and be set in config.c Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-30Merge branch 'cb/khash-maybe-unused-function'Libravatar Junio C Hamano1-0/+2
Build fix. * cb/khash-maybe-unused-function: khash: silence -Wunused-function for delta-islands commit-slabs: move MAYBE_UNUSED out
2018-10-26Merge branch 'js/mingw-default-ident'Libravatar Junio C Hamano1-0/+4
The logic to select the default user name and e-mail on Windows has been improved. * js/mingw-default-ident: mingw: use domain information for default email getpwuid(mingw): provide a better default for the user name getpwuid(mingw): initialize the structure only once
2018-10-24commit-slabs: move MAYBE_UNUSED outLibravatar Carlo Marcelo Arenas Belón1-0/+2
after 36da893114 ("config.mak.dev: enable -Wunused-function", 2018-10-18) it is expected to be used to prevent -Wunused-function warnings for code that was macro generated Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16mingw: use domain information for default emailLibravatar Johannes Schindelin1-0/+4
When a user is registered in a Windows domain, it is really easy to obtain the email address. So let's do that. Suggested by Lutz Roeder. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-04mingw: bump the minimum Windows version to VistaLibravatar Johannes Schindelin1-1/+1
Quite some time ago, a last plea to the XP users out there who want to see Windows XP support in Git for Windows, asking them to get engaged and help, vanished into the depths of the universe. We tried for a long time to play nice with the last remaining XP users who somehow manage to build Git from source, but a recent update of mingw-w64 (7.0.0.5233.e0c09544 -> 7.0.0.5245.edf66197) finally dropped the last sign of XP support, and Git for Windows' SDK is no longer able to build core Git's `master` branch as a consequence. (Git for Windows' `master` branch already bumped the minimum Windows version to Vista a while ago, so it is fine.) It is time to require Windows Vista or later to build Git from source. This, incidentally, lets us use quite a few nice new APIs. It also means that we no longer need the inet_pton() and inet_ntop() emulation, which is nice. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-04mingw: set _WIN32_WINNT explicitly for Git for WindowsLibravatar Johannes Schindelin1-1/+1
Previously, we only ever declared a target Windows version if compiling with Visual C. Which meant that we were relying on the MinGW headers to guess which Windows version we want to target... Let's be explicit about it, in particular because we actually want to bump the target Windows version to Vista (which we will do in the next commit). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-20Merge branch 'js/typofixes'Libravatar Junio C Hamano1-1/+1
Comment update. * js/typofixes: remote-curl: remove spurious period git-compat-util.h: fix typo
2018-08-08git-compat-util.h: fix typoLibravatar Johannes Schindelin1-1/+1
The words "save" and "safe" are both very wonderful words, each with their own set of meanings. Let's not confuse them with one another save on occasion of a pun. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-26automatically ban strcpy()Libravatar Jeff King1-0/+6
There are a few standard C functions (like strcpy) which are easy to misuse. E.g.: char path[PATH_MAX]; strcpy(path, arg); may overflow the "path" buffer. Sometimes there's an earlier constraint on the size of "arg", but even in such a case it's hard to verify that the code is correct. If the size really is unbounded, you're better off using a dynamic helper like strbuf: struct strbuf path = STRBUF_INIT; strbuf_addstr(path, arg); or if it really is bounded, then use xsnprintf to show your expectation (and get a run-time assertion): char path[PATH_MAX]; xsnprintf(path, sizeof(path), "%s", arg); which makes further auditing easier. We'd usually catch undesirable code like this in a review, but there's no automated enforcement. Adding that enforcement can help us be more consistent and save effort (and a round-trip) during review. This patch teaches the compiler to report an error when it sees strcpy (and will become a model for banning a few other functions). This has a few advantages over a separate linting tool: 1. We know it's run as part of a build cycle, so it's hard to ignore. Whereas an external linter is an extra step the developer needs to remember to do. 2. Likewise, it's basically free since the compiler is parsing the code anyway. 3. We know it's robust against false positives (unlike a grep-based linter). The two big disadvantages are: 1. We'll only check code that is actually compiled, so it may miss code that isn't triggered on your particular system. But since presumably people don't add new code without compiling it (and if they do, the banned function list is the least of their worries), we really only care about failing to clean up old code when adding new functions to the list. And that's easy enough to address with a manual audit when adding a new function (which is what I did for the functions here). 2. If this ends up generating false positives, it's going to be harder to disable (as opposed to a separate linter, which may have mechanisms for overriding a particular case). But the intent is to only ban functions which are obviously bad, and for which we accept using an alternative even when this particular use isn't buggy (e.g., the xsnprintf alternative above). The implementation here is simple: we'll define a macro for the banned function which replaces it with a reference to a descriptively named but undeclared identifier. Replacing it with any invalid code would work (since we just want to break compilation). But ideally we'd meet these goals: - it should be portable; ideally this would trigger everywhere, and does not need to be part of a DEVELOPER=1 setup (because unlike warnings which may depend on the compiler or system, this is a clear indicator of something wrong in the code). - it should generate a readable error that gives the developer a clue what happened - it should avoid generating too much other cruft that makes it hard to see the actual error - it should mention the original callsite in the error The output with this patch looks like this (using gcc 7, on a checkout with 022d2ac1f3 reverted, which removed the final strcpy from blame.c): CC builtin/blame.o In file included from ./git-compat-util.h:1246, from ./cache.h:4, from builtin/blame.c:8: builtin/blame.c: In function ‘cmd_blame’: ./banned.h:11:22: error: ‘sorry_strcpy_is_a_banned_function’ undeclared (first use in this function) #define BANNED(func) sorry_##func##_is_a_banned_function ^~~~~~ ./banned.h:14:21: note: in expansion of macro ‘BANNED’ #define strcpy(x,y) BANNED(strcpy) ^~~~~~ builtin/blame.c:1074:4: note: in expansion of macro ‘strcpy’ strcpy(repeated_meta_color, GIT_COLOR_CYAN); ^~~~~~ ./banned.h:11:22: note: each undeclared identifier is reported only once for each function it appears in #define BANNED(func) sorry_##func##_is_a_banned_function ^~~~~~ ./banned.h:14:21: note: in expansion of macro ‘BANNED’ #define strcpy(x,y) BANNED(strcpy) ^~~~~~ builtin/blame.c:1074:4: note: in expansion of macro ‘strcpy’ strcpy(repeated_meta_color, GIT_COLOR_CYAN); ^~~~~~ This prominently shows the phrase "strcpy is a banned function", along with the original callsite in blame.c and the location of the ban code in banned.h. Which should be enough to get even a developer seeing this for the first time pointed in the right direction. This doesn't match our ideals perfectly, but it's a pretty good balance. A few alternatives I tried: 1. Instead of using an undeclared variable, using an undeclared function. This shortens the message, because the "each undeclared identifier" message is not needed (and as you can see above, it triggers a separate mention of each of the expansion points). But it doesn't actually stop compilation unless you use -Werror=implicit-function-declaration in your CFLAGS. This is the case for DEVELOPER=1, but not for a default build (on the other hand, we'd eventually produce a link error pointing to the correct source line with the descriptive name). 2. The linux kernel uses a similar mechanism in its BUILD_BUG_ON_MSG(), where they actually declare the function but do so with gcc's error attribute. But that's not portable to other compilers (and it also runs afoul of our error() macro). We could make a gcc-specific technique and fallback on other compilers, but it's probably not worth the complexity. It also isn't significantly shorter than the error message shown above. 3. We could drop the BANNED() macro, which would shorten the number of lines in the error. But curiously, removing it (and just expanding strcpy directly to the bogus identifier) causes gcc _not_ to report the original line of code. So this strategy seems to be an acceptable mix of information, portability, simplicity, and robustness, without _too_ much extra clutter. I also tested it with clang, and it looks as good (actually, slightly less cluttered than with gcc). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-30Merge branch 'js/use-bug-macro'Libravatar Junio C Hamano1-1/+4
Developer support update, by using BUG() macro instead of die() to mark codepaths that should not happen more clearly. * js/use-bug-macro: BUG_exit_code: fix sparse "symbol not declared" warning Convert remaining die*(BUG) messages Replace all die("BUG: ...") calls by BUG() ones run-command: use BUG() to report bugs, not die() test-tool: help verifying BUG() code paths
2018-05-29Sync with Git 2.17.1Libravatar Junio C Hamano1-0/+17
* maint: (25 commits) Git 2.17.1 Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 fsck: complain when .gitmodules is a symlink index-pack: check .gitmodules files with --strict unpack-objects: call fsck_finish() after fscking objects fsck: call fsck_finish() after fscking objects fsck: check .gitmodules content fsck: handle promisor objects in .gitmodules check fsck: detect gitmodules files fsck: actually fsck blob data fsck: simplify ".git" check index-pack: make fsck error message more specific verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant ...
2018-05-23Merge branch 'nd/repack-keep-pack'Libravatar Junio C Hamano1-0/+4
"git gc" in a large repository takes a lot of time as it considers to repack all objects into one pack by default. The command has been taught to pretend as if the largest existing packfile is marked with ".keep" so that it is left untouched while objects in other packs and loose ones are repacked. * nd/repack-keep-pack: pack-objects: show some progress when counting kept objects gc --auto: exclude base pack if not enough mem to "repack -ad" gc: handle a corner case in gc.bigPackThreshold gc: add gc.bigPackThreshold config gc: add --keep-largest-pack option repack: add --keep-pack option t7700: have closing quote of a test at the beginning of line
2018-05-22Sync with Git 2.16.4Libravatar Junio C Hamano1-0/+17
* maint-2.16: Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.15.2Libravatar Junio C Hamano1-0/+17
* maint-2.15: Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.14.4Libravatar Junio C Hamano1-0/+17
* maint-2.14: Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.13.7Libravatar Junio C Hamano1-0/+17
* maint-2.13: Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-21skip_prefix: add case-insensitive variantLibravatar Jeff King1-0/+17
We have the convenient skip_prefix() helper, but if you want to do case-insensitive matching, you're stuck doing it by hand. We could add an extra parameter to the function to let callers ask for this, but the function is small and somewhat performance-critical. Let's just re-implement it for the case-insensitive version. Signed-off-by: Jeff King <peff@peff.net>
2018-05-10BUG_exit_code: fix sparse "symbol not declared" warningLibravatar Ramsay Jones1-0/+3
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-08Merge branch 'ls/checkout-encoding'Libravatar Junio C Hamano1-0/+1
The new "checkout-encoding" attribute can ask Git to convert the contents to the specified encoding when checking out to the working tree (and the other way around when checking in). * ls/checkout-encoding: convert: add round trip check based on 'core.checkRoundtripEncoding' convert: add tracing for 'working-tree-encoding' attribute convert: check for detectable errors in UTF encodings convert: add 'working-tree-encoding' attribute utf8: add function to detect a missing UTF-16/32 BOM utf8: add function to detect prohibited UTF-16/32 BOM utf8: teach same_encoding() alternative UTF encoding names strbuf: add a case insensitive starts_with() strbuf: add xstrdup_toupper() strbuf: remove unnecessary NUL assignment in xstrdup_tolower()