summaryrefslogtreecommitdiff
path: root/compat
AgeCommit message (Collapse)AuthorFilesLines
2014-07-15Win32: Unicode file name support (dirent)Libravatar Karsten Blees2-21/+11
Changes opendir/readdir to use Windows Unicode APIs and convert between UTF-8/UTF-16. Removes parameter checks that are already covered by xutftowcs_path. This changes detection of ENAMETOOLONG from MAX_PATH - 2 to MAX_PATH (matching is_dir_empty in mingw.c). If name + "/*" or the resulting absolute path is too long, FindFirstFile fails and errno is set through err_win_to_posix. Increases the size of dirent.d_name to accommodate the full WIN32_FIND_DATA.cFileName converted to UTF-8 (UTF-16 to UTF-8 conversion may grow by factor three in the worst case). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-15Win32: Unicode file name support (except dirent)Libravatar Karsten Blees2-56/+160
Replaces Windows "ANSI" APIs dealing with file- or path names with their Unicode equivalent, adding UTF-8/UTF-16LE conversion as necessary. The dirent API (opendir/readdir/closedir) is updated in a separate commit. Adds trivial wrappers for access, chmod and chdir. Adds wrapper for mktemp (needed for both mkstemp and mkdtemp). The simplest way to convert a repository with legacy-encoded (e.g. Cp1252) file names to UTF-8 ist to checkout with an old msysgit version and "git add --all & git commit" with the new version. Includes a fix for bug reported by John Chen: On Windows XP (not Win7), directories cannot be deleted while a find handle is open, causing "Deletion of directory '...' failed. Should I try again?" prompts. Prior to this commit, these failures were silently ignored due to strbuf_free in is_dir_empty resetting GetLastError to ERROR_SUCCESS. Close the find handle in is_dir_empty so that git doesn't block deletion of the directory even after all other applications have released it. Reported-by: John Chen <john0312@gmail.com> Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-09Merge branch 'sk/mingw-unicode-spawn-args'Libravatar Junio C Hamano1-23/+71
* sk/mingw-unicode-spawn-args: Win32: Unicode arguments (incoming) Win32: Unicode arguments (outgoing) MinGW: disable CRT command line globbing Win32: fix potential multi-threading issue Win32: simplify internal mingw_spawn* APIs Win32: let mingw_execve() return an int
2014-07-09Merge branch 'sk/mingw-dirent'Libravatar Junio C Hamano2-67/+57
* sk/mingw-dirent: Win32 dirent: improve dirent implementation Win32 dirent: clarify #include directives Win32 dirent: change FILENAME_MAX to MAX_PATH Win32 dirent: remove unused dirent.d_reclen member Win32 dirent: remove unused dirent.d_ino member
2014-07-09Merge branch 'sk/mingw-uni-console'Libravatar Junio C Hamano3-123/+533
* sk/mingw-uni-console: Win32: reliably detect console pipe handles Win32: fix broken pipe detection Win32: Thread-safe windows console output Win32: add Unicode conversion functions Win32: warn if the console font doesn't support Unicode Win32: detect console streams more reliably Win32: support Unicode console output
2014-07-09Merge branch 'sk/mingw-main'Libravatar Junio C Hamano2-10/+19
* sk/mingw-main: mingw: avoid const warning Win32: move main macro to a function
2014-06-16Win32: Unicode arguments (incoming)Libravatar Karsten Blees1-2/+40
Convert command line arguments from UTF-16 to UTF-8 on startup. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Win32: Unicode arguments (outgoing)Libravatar Karsten Blees1-4/+14
Convert command line arguments from UTF-8 to UTF-16 when creating other processes. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16MinGW: disable CRT command line globbingLibravatar Karsten Blees1-0/+6
MingwRT listens to _CRT_glob to decide if __getmainargs should perform globbing, with the default being that it should. Unfortunately, __getmainargs globbing is sub-par; for instance patterns like "*.c" will only match c-sources in the current directory. Disable __getmainargs' command line wildcard expansion, so these patterns will be left untouched, and handled by Git's superior built-in globbing instead. MSVC defaults to no globbing, so we don't need to do anything in that case. This fixes t5505 and t7810. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Win32: fix potential multi-threading issueLibravatar Karsten Blees1-1/+1
...by removing a static buffer in do_stat_internal. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Win32: simplify internal mingw_spawn* APIsLibravatar Karsten Blees1-15/+8
The only public spawn function that needs to tweak the environment is mingw_spawnvpe (called from start_command). Nevertheless, all internal spawn* functions take an env parameter and needlessly pass the global char **environ around. Remove the env parameter where it's not needed. This removes the internal mingw_execve abstraction, which is no longer needed. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Win32: let mingw_execve() return an intLibravatar Johannes Schindelin1-2/+3
This is in the great tradition of POSIX. Original fix by Olivier Refalo. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Win32: reliably detect console pipe handlesLibravatar Karsten Blees1-18/+7
As of "Win32: Thread-safe windows console output", child processes may print to the console even if stdout has been redirected to a file. E.g.: git config tar.cat.command "cat" git archive -o test.cat HEAD Detecting whether stdout / stderr point to our console pipe is currently based on the assumption that OS HANDLE values are never reused. This is apparently not true if stdout / stderr is replaced via dup2() (as in builtin/archive.c:17). Instead of comparing handle values, check if the file descriptor isatty() backed by a pipe OS handle. This is only possible by swapping the handles in MSVCRT's internal data structures, as we do in winansi_init(). Reported-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Merge branch 'cb/byte-order'Libravatar Junio C Hamano1-9/+24
Compatibility enhancement for Solaris. * cb/byte-order: compat/bswap.h: fix endianness detection compat/bswap.h: restore preference __BIG_ENDIAN over BIG_ENDIAN compat/bswap.h: detect endianness on more platforms that don't use BYTE_ORDER
2014-06-16Merge branch 'sk/windows-unc-path'Libravatar Junio C Hamano2-0/+26
* sk/windows-unc-path: Windows: allow using UNC path for git repository
2014-06-10Win32: fix broken pipe detectionLibravatar Karsten Blees2-46/+70
As of "Win32: Thread-safe windows console output", git-log no longer terminates when the pager process dies. This is due to disabling buffering for the replaced stdout / stderr streams. Git-log will periodically fflush stdout (see write_or_die.c/mayble_flush_or_die()), but with no buffering, this is a NOP that always succeeds (so we never detect the EPIPE error). Exchange the original console handles with our console thread pipe handles by accessing the internal MSVCRT data structures directly (which are exposed via __pioinfo for some reason). Implement this with minimal assumptions about the actual data structure to make it work with different (hopefully even future) MSVCRT versions. While messing with internal data structures is ugly, this patch solves the problem at the source instead of adding more workarounds. We no longer need the special winansi_isatty override, and the limitations documented in "Win32: Thread-safe windows console output" are gone (i.e. fdopen(1/2) returns unbuffered streams now, and isatty() for duped console file descriptors works as expected). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Win32: Thread-safe windows console outputLibravatar Karsten Blees3-149/+273
Winansi.c has many static variables that are accessed and modified from the [v][f]printf / fputs functions overridden in the file. This may cause multi threaded git commands that print to the console to produce corrupted output or even crash. Additionally, winansi.c doesn't override all functions that can be used to print to the console (e.g. fwrite, write, fputc are missing), so that ANSI escapes don't work properly for some git commands (e.g. git-grep). Instead of doing ANSI emulation in just a few wrapped functions on top of the IO API, let's plug into the IO system and take advantage of the thread safety inherent to the IO system. Redirect stdout and stderr to a pipe if they point to the console. A background thread reads from the pipe, handles ANSI escape sequences and UTF-8 to UTF-16 conversion, then writes to the console. The pipe-based stdout and stderr replacements must be set to unbuffered, as MSVCRT doesn't support line buffering and fully buffered streams are inappropriate for console output. Due to the byte-oriented pipe, ANSI escape sequences and multi-byte UTF-8 sequences can no longer be expected to arrive in one piece. Replace the string-based ansi_emulate() with a simple stateful parser (this also fixes colored diff hunk headers, which were broken as of commit 2efcc977). Override isatty to return true for the pipes redirecting to the console. Exec/spawn obtain the original console handle to pass to the next process via winansi_get_osfhandle(). All other overrides are gone, the default stdio implementations work as expected with the piped stdout/stderr descriptors. Global variables are either initialized on startup (single threaded) or exclusively modified by the background thread. Threads communicate through the pipe, no further synchronization is necessary. The background thread is terminated by disonnecting the pipe after flushing the stdio and pipe buffers. This doesn't work for anonymous pipes (created via CreatePipe), as DisconnectNamedPipe only works on the read end, which discards remaining data. Thus we have to setup the pipe manually, with the write end beeing the server (opened with CreateNamedPipe) and the read end the client (opened with CreateFile). Limitations: doesn't track reopened or duped file descriptors, i.e.: - fdopen(1/2) returns fully buffered streams - dup(1/2), dup2(1/2) returns normal pipe descriptors (i.e. isatty() = false, winansi_get_osfhandle won't return the original console handle) Currently, only the git-format-patch command uses xfdopen(xdup(1)) (see "realstdout" in builtin/log.c), but works well with these limitations. Many thanks to Atsushi Nakagawa <atnak@chejz.com> for suggesting and reviewing the thread-exit-mechanism. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Win32: add Unicode conversion functionsLibravatar Karsten Blees2-0/+189
Add Unicode conversion functions to convert between Windows native UTF-16LE encoding to UTF-8 and back. To support repositories with legacy-encoded file names, the UTF-8 to UTF-16 conversion function tries to create valid, unique file names even for invalid UTF-8 byte sequences, so that these repositories can be checked out without error. The current implementation leaves invalid UTF-8 bytes in range 0xa0 - 0xff as is (producing printable Unicode chars \u00a0 - \u00ff, equivalent to ISO-8859-1), and converts 0x80 - 0x9f to hex-code (\u0080 - \u009f are control chars). The Windows MultiByteToWideChar API was not used as it either drops invalid UTF-8 sequences (on Win2k/XP; producing non-unique or even empty file names) or converts them to the replacement char \ufffd (Vista/7; causing ERROR_INVALID_NAME in subsequent calls to file system APIs). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Win32: warn if the console font doesn't support UnicodeLibravatar Karsten Blees1-0/+66
Unicode console output won't display correctly with default settings because the default console font ("Terminal") only supports the system's OEM charset. Unfortunately, this is a user specific setting, so it cannot be easily fixed by e.g. some registry tricks in the setup program. This change prints a warning on exit if console output contained non-ascii characters and the console font is supposedly not a TrueType font (which usually have decent Unicode support). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Win32: detect console streams more reliablyLibravatar Karsten Blees1-24/+26
GetStdHandle(STD_OUTPUT_HANDLE) doesn't work for stderr if stdout is redirected. Use _get_osfhandle of the FILE* instead. _isatty() is true for all character devices (including parallel and serial ports). Check return value of GetConsoleScreenBufferInfo instead to reliably detect console handles (also don't initialize internal state from an uninitialized CONSOLE_SCREEN_BUFFER_INFO structure if the function fails). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Win32: support Unicode console outputLibravatar Karsten Blees2-6/+22
WriteConsoleW seems to be the only way to reliably print unicode to the console (without weird code page conversions). Also redirects vfprintf to the winansi.c version. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10mingw: avoid const warningLibravatar Stepan Kasal1-1/+1
Fix const warnings in http-fetch.c and remote-curl.c main() where is argv declared as const. The fix should work for all future declarations of main, no matter whether the second parameter's type is "char**", "const char**", or "char *[]". Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Win32: move main macro to a functionLibravatar Karsten Blees2-10/+19
The code in the MinGW main macro is getting more and more complex, move to a separate initialization function for readabiliy and extensibility. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10Windows: allow using UNC path for git repositoryLibravatar Cezary Zawadka2-0/+26
[efl: moved MinGW-specific part to compat/] [jes: fixed compilation on non-Windows] Eric Sunshine fixed mingw_offset_1st_component() to return consistently "foo" for UNC "//machine/share/foo", cf http://groups.google.com/group/msysgit/browse_thread/thread/c0af578549b5dda0 Author: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Cezary Zawadka <czawadka@gmail.com> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09Win32 dirent: improve dirent implementationLibravatar Karsten Blees1-59/+54
Improve the dirent implementation by removing the relics that were once necessary to plug into the now unused MinGW runtime, in preparation for Unicode file name support. Move FindFirstFile to opendir, and FindClose to closedir, with the following implications: - DIR.dd_name is no longer needed - chdir(one); opendir(relative); chdir(two); readdir() works as expected (i.e. lists one/relative instead of two/relative) - DIR.dd_handle is a valid handle for the entire lifetime of the DIR struct - thus, all checks for dd_handle == INVALID_HANDLE_VALUE and dd_handle == 0 have been removed - the special case that the directory has been fully read (which was previously explicitly tracked with dd_handle == INVALID_HANDLE_VALUE && dd_stat != 0) is now handled implicitly by the FindNextFile error handling code (if a client continues to call readdir after receiving NULL, FindNextFile will continue to fail with ERROR_NO_MORE_FILES, to the same effect) - extracting dirent data from WIN32_FIND_DATA is needed in two places, so moved to its own method - GetFileAttributes is no longer needed. The same information can be obtained from the FindFirstFile error code, which is ERROR_DIRECTORY if the name is NOT a directory (-> ENOTDIR), otherwise we can use err_win_to_posix (e.g. ERROR_PATH_NOT_FOUND -> ENOENT). The ERROR_DIRECTORY case could be fixed in err_win_to_posix, but this probably breaks other functionality. Removes the ERROR_NO_MORE_FILES check after FindFirstFile (this was fortunately a NOOP (searching for '*' always finds '.' and '..'), otherwise the subsequent code would have copied data from an uninitialized buffer). Changes malloc to git support function xmalloc, so opendir will die() if out of memory, rather than failing with ENOMEM and letting git work on incomplete directory listings (error handling in dir.c is quite sparse). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09Win32 dirent: clarify #include directivesLibravatar Karsten Blees1-2/+1
Git-compat-util.h is two dirs up, and already includes <dirent.h> (which is the same as "dirent.h" due to -Icompat/win32 in the Makefile). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09Win32 dirent: change FILENAME_MAX to MAX_PATHLibravatar Karsten Blees1-1/+1
FILENAME_MAX and MAX_PATH are both 260 on Windows, however, MAX_PATH is used throughout the other Win32 code in Git, and also defines the length of file name buffers in the Win32 API (e.g. WIN32_FIND_DATA.cFileName, from which we're copying the dirent data). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09Win32 dirent: remove unused dirent.d_reclen memberLibravatar Karsten Blees1-4/+1
Remove the union around dirent.d_type and the unused dirent.d_reclen member (which was necessary for compatibility with the MinGW dirent runtime, which is no longer used). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09Win32 dirent: remove unused dirent.d_ino memberLibravatar Karsten Blees1-1/+0
There are no proper inodes on Windows, so remove dirent.d_ino and #define NO_D_INO_IN_DIRENT in the Makefile (this skips e.g. an ineffective qsort in fsck.c). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06Merge branch 'tl/relax-in-poll-emulation'Libravatar Junio C Hamano1-1/+1
* tl/relax-in-poll-emulation: compat/poll: sleep 1 millisecond to avoid busy wait
2014-06-03Merge branch 'ym/fix-opportunistic-index-update-race'Libravatar Junio C Hamano1-3/+1
Read-only operations such as "git status" that internally refreshes the index write out the refreshed index to the disk to optimize future accesses to the working tree, but this could race with a "read-write" operation that modify the index while it is running. Detect such a race and avoid overwriting the index. Duy raised a good point that we may need to do the same for the normal writeout codepath, not just the "opportunistic" update codepath. While that is true, nobody sane would be running two simultaneous operations that are clearly write-oriented competing with each other against the same index file. So in that sense that can be done as a less urgent follow-up for this topic. * ym/fix-opportunistic-index-update-race: read-cache.c: verify index file before we opportunistically update it wrapper.c: add xpread() similar to xread()
2014-06-03Merge branch 'ks/tree-diff-nway'Libravatar Junio C Hamano1-0/+0
Instead of running N pair-wise diff-trees when inspecting a N-parent merge, find the set of paths that were touched by walking N+1 trees in parallel. These set of paths can then be turned into N pair-wise diff-tree results to be processed through rename detections and such. And N=2 case nicely degenerates to the usual 2-way diff-tree, which is very nice. * ks/tree-diff-nway: mingw: activate alloca combine-diff: speed it up, by using multiparent diff tree-walker directly tree-diff: rework diff_tree() to generate diffs for multiparent cases as well Portable alloca for Git tree-diff: reuse base str(buf) memory on sub-tree recursion tree-diff: no need to call "full" diff_tree_sha1 from show_path() tree-diff: rework diff_tree interface to be sha1 based tree-diff: diff_tree() should now be static tree-diff: remove special-case diff-emitting code for empty-tree cases tree-diff: simplify tree_entry_pathcmp tree-diff: show_path prototype is not needed anymore tree-diff: rename compare_tree_entry -> tree_entry_pathcmp tree-diff: move all action-taking code out of compare_tree_entry() tree-diff: don't assume compare_tree_entry() returns -1,0,1 tree-diff: consolidate code for emitting diffs and recursion in one place tree-diff: show_tree() is not needed tree-diff: no need to pass match to skip_uninteresting() tree-diff: no need to manually verify that there is no mode change for a path combine-diff: move changed-paths scanning logic into its own function combine-diff: move show_log_first logic/action out of paths scanning
2014-05-30compat/bswap.h: fix endianness detectionLibravatar Ben Walton1-1/+1
The changes to make detection of endianness more portable had a bug that breaks on (at least) Solaris x86. The bug appears to be a simple copy/paste typo. It checks for _BIG_ENDIAN and not _LITTLE_ENDIAN for both the case where we would decide the system is big endian and little endian. Instead, the second test should be for _LITTLE_ENDIAN and not _BIG_ENDIAN. Two fixes were possible: 1. Change the negation order of the conditions in the second test. 2. Reverse the order of the conditions in the second test. Use the second option so that the condition we expect is always a positive check. Signed-off-by: Ben Walton <bdwalton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-02compat/bswap.h: restore preference __BIG_ENDIAN over BIG_ENDIANLibravatar Junio C Hamano1-7/+7
The previous commit swaps the order we check the macros defined by the compiler and the system headers from the original. Since the order of check should not matter (i.e. it is insane to define both __BIG_ENDIAN and friends and BIG_ENDIAN and friends and in a conflicting way), it is the most conservative thing to do not to change it. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-02compat/bswap.h: detect endianness on more platforms that don't use BYTE_ORDERLibravatar Charles Bailey1-9/+24
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-29compat/poll: sleep 1 millisecond to avoid busy waitLibravatar Theodore Leblond1-1/+1
SwitchToThread() only gives away the rest of the current time slice to another thread in the current process. So if the thread that feeds the file decscriptor we're polling is not in the current process, we get busy-waiting. I played around with this quite a bit. After trying some more complex schemes, I found that what worked best is to just sleep 1 millisecond between iterations. Though it's a very short time, it still completely eliminates the busy wait condition, without hurting perf. There code uses SleepEx(1, TRUE) to sleep. See this page for a good discussion of why that is better than calling SwitchToThread, which is what was used previously: http://stackoverflow.com/questions/1383943/switchtothread-vs-sleep1 Note that calling SleepEx(0, TRUE) does *not* solve the busy wait. The most striking case was when testing on a UNC share with a large repo, on a single CPU machine. Without the fix, it took 4 minutes 15 seconds, and with the fix it took just 1:08! I think it's because git-upload-pack's busy wait was eating the CPU away from the git process that's doing the real work. With multi-proc, the timing is not much different, but tons of CPU time is still wasted, which can be a killer on a server that needs to do bunch of other things. I also tested the very fast local case, and didn't see any measurable difference. On a big repo with 4500 files, the upload-pack took about 2 seconds with and without the fix. [jc: this was first accepted in msysgit tree in May 2012 via a pull request and Paolo Bonzini has also accepted the same fix to Gnulib around the same time; see $gmane/247518 for a bit more detail] Signed-off-by: Stepan Kasal <kasal@ucw.cz> Acked-by: Johannes Sixt <j6t@kdbg.org> Acked-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-10wrapper.c: add xpread() similar to xread()Libravatar Yiannis Marangos1-3/+1
It is a common mistake to call read(2)/pread(2) and forget to anticipate that they may return error with EAGAIN/EINTR when the system call is interrupted. We have xread() helper to relieve callers of read(2) from having to worry about it; add xpread() helper to do the same for pread(2). Update the caller in the builtin/index-pack.c and the mmap emulation in compat/. Signed-off-by: Yiannis Marangos <yiannis.marangos@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-09Merge branch 'jk/commit-dates-parsing-fix' into maintLibravatar Junio C Hamano1-0/+29
* jk/commit-dates-parsing-fix: t4212: loosen far-in-future test for AIX date: recognize bogus FreeBSD gmtime output
2014-04-09mingw: activate allocaLibravatar Kirill Smelkov1-0/+0
Both MSVC and MINGW have alloca(3) definitions in malloc.h, so by moving win32-compat alloca.h from compat/vcbuild/include/ to compat/win32/ , which is included by both MSVC and MINGW CFLAGS, we can make alloca() work on both those Windows environments. In MINGW, malloc.h has explicit check for GNUC and if it is so, defines alloca to __builtin_alloca, so it looks like we don't need to add any code to here-shipped alloca.h to get optimum performance. Compile-tested on Windows in MSysGit. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Acked-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-08Merge branch 'jk/commit-dates-parsing-fix'Libravatar Junio C Hamano1-0/+29
Finishing touches for portability. * jk/commit-dates-parsing-fix: t4212: loosen far-in-future test for AIX date: recognize bogus FreeBSD gmtime output
2014-04-01date: recognize bogus FreeBSD gmtime outputLibravatar Jeff King1-0/+29
Most gmtime implementations return a NULL value when they encounter an error (and this behavior is specified by ANSI C and POSIX). FreeBSD's implementation, however, will simply leave the "struct tm" untouched. Let's also recognize this and convert it to a NULL (with this patch, t4212 should pass on FreeBSD). Reported-by: René Scharfe <l.s.r@web.de> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-31Merge branch 'mr/msvc-link-with-lcurl'Libravatar Junio C Hamano1-0/+2
* mr/msvc-link-with-lcurl: MSVC: allow linking with the cURL library
2014-03-27MSVC: allow linking with the cURL libraryLibravatar Marat Radchenko1-0/+2
Teach the clink.pl script that -lcurl is a request to link with the cURL library, and drop NO_CURL from config.mak.uname for the MSVC platform. Signed-off-by: Marat Radchenko <marat@slonopotamus.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-14Merge branch 'nd/no-more-fnmatch'Libravatar Junio C Hamano2-578/+0
We started using wildmatch() in place of fnmatch(3); complete the process and stop using fnmatch(3). * nd/no-more-fnmatch: actually remove compat fnmatch source code stop using fnmatch (either native or compat) Revert "test-wildmatch: add "perf" command to compare wildmatch and fnmatch" use wildmatch() directly without fnmatch() wrapper
2014-02-27Merge branch 'jk/pack-bitmap'Libravatar Junio C Hamano1-1/+111
Borrow the bitmap index into packfiles from JGit to speed up enumeration of objects involved in a commit range without having to fully traverse the history. * jk/pack-bitmap: (26 commits) ewah: unconditionally ntohll ewah data ewah: support platforms that require aligned reads read-cache: use get_be32 instead of hand-rolled ntoh_l block-sha1: factor out get_be and put_be wrappers do not discard revindex when re-preparing packfiles pack-bitmap: implement optional name_hash cache t/perf: add tests for pack bitmaps t: add basic bitmap functionality tests count-objects: recognize .bitmap in garbage-checking repack: consider bitmaps when performing repacks repack: handle optional files created by pack-objects repack: turn exts array into array-of-struct repack: stop using magic number for ARRAY_SIZE(exts) pack-objects: implement bitmap writing rev-list: add bitmap mode to speed up object lists pack-objects: use bitmaps when packing objects pack-objects: split add_object_entry pack-bitmap: add support for bitmap indexes documentation: add documentation for the bitmap format ewah: compressed bitmap implementation ...
2014-02-20actually remove compat fnmatch source codeLibravatar Nguyễn Thái Ngọc Duy2-578/+0
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-27Merge branch 'ef/mingw-write'Libravatar Junio C Hamano2-20/+0
* ef/mingw-write: mingw: remove mingw_write prefer xwrite instead of write
2014-01-23read-cache: use get_be32 instead of hand-rolled ntoh_lLibravatar Jeff King1-0/+4
Commit d60c49c (read-cache.c: allow unaligned mapping of the index file, 2012-04-03) introduced helpers to access unaligned data. However, we already have get_be32, which has a few advantages: 1. It's already written, so we avoid duplication. 2. It's probably faster, since it does the endian conversion and the alignment fix at the same time. 3. The get_be32 code is well-tested, having been in block-sha1 for a long time. By contrast, our custom helpers were probably almost never used, since the user needed to manually define a macro to enable them. We have to add a get_be16 implementation to the existing get_be32, but that is very simple to do. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-23block-sha1: factor out get_be and put_be wrappersLibravatar Jeff King1-0/+32
The BLK_SHA1 code has optimized wrappers for doing endian conversions on memory that may not be aligned. Let's pull them out so that we can use them elsewhere, especially the time-tested list of platforms that prefer each strategy. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-17mingw: remove mingw_writeLibravatar Erik Faye-Lund2-20/+0
Since 0b6806b9 ("xread, xwrite: limit size of IO to 8MB"), this wrapper is no longer needed, as read and write are already split into small chunks. Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>