summaryrefslogtreecommitdiff
path: root/diff.c
AgeCommit message (Collapse)AuthorFilesLines
2015-08-31Merge branch 'hv/submodule-config'Libravatar Junio C Hamano1-0/+1
The gitmodules API accessed from the C code learned to cache stuff lazily. * hv/submodule-config: submodule: allow erroneous values for the fetchRecurseSubmodules option submodule: use new config API for worktree configurations submodule: extract functions for config set and lookup submodule: implement a config API for lookup of .gitmodules values
2015-08-25Merge branch 'mh/tempfile'Libravatar Junio C Hamano1-23/+23
The "lockfile" API has been rebuilt on top of a new "tempfile" API. * mh/tempfile: credential-cache--daemon: use tempfile module credential-cache--daemon: delete socket from main() gc: use tempfile module to handle gc.pid file lock_repo_for_gc(): compute the path to "gc.pid" only once diff: use tempfile module setup_temporary_shallow(): use tempfile module write_shared_index(): use tempfile module register_tempfile(): new function to handle an existing temporary file tempfile: add several functions for creating temporary files prepare_tempfile_object(): new function, extracted from create_tempfile() tempfile: a new module for handling temporary files commit_lock_file(): use get_locked_file_path() lockfile: add accessor get_lock_file_path() lockfile: add accessors get_lock_file_fd() and get_lock_file_fp() create_bundle(): duplicate file descriptor to avoid closing it twice lockfile: move documentation to lockfile.h and lockfile.c
2015-08-19submodule: use new config API for worktree configurationsLibravatar Heiko Voigt1-0/+1
We remove the extracted functions and directly parse into and read out of the cache. This allows us to have one unified way of accessing submodule configuration values specific to single submodules. Regardless whether we need to access a configuration from history or from the worktree. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-12diff: use tempfile moduleLibravatar Michael Haggerty1-23/+23
Also add some code comments explaining how the fields in "struct diff_tempfile" are used. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-03Merge branch 'dt/log-follow-config'Libravatar Junio C Hamano1-2/+3
Add a new configuration variable to enable "--follow" automatically when "git log" is run with one pathspec argument. * dt/log-follow-config: log: add "log.follow" configuration variable
2015-07-15Merge branch 'jc/diff-ws-error-highlight'Libravatar Junio C Hamano1-1/+6
A hotfix to a new feature in 2.5.0-rc. * jc/diff-ws-error-highlight: diff: parse ws-error-highlight option more strictly
2015-07-12diff: parse ws-error-highlight option more strictlyLibravatar René Scharfe1-1/+6
Check if a matched token is followed by a delimiter before advancing the pointer arg. This avoids accepting composite words like "allnew" or "defaultcontext" and misparsing them as "new" or "context". Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-07-09log: add "log.follow" configuration variableLibravatar David Turner1-2/+3
People who work on projects with mostly linear history with frequent whole file renames may want to always use "git log --follow" when inspecting the life of the content that live in a single path. Teach the command to behave as if "--follow" was given from the command line when log.follow configuration variable is set *and* there is one (and only one) path on the command line. Signed-off-by: David Turner <dturner@twopensource.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-11Merge branch 'jk/color-diff-plain-is-context'Libravatar Junio C Hamano1-15/+15
"color.diff.plain" was a misnomer; give it 'color.diff.context' as a more logical synonym. * jk/color-diff-plain-is-context: diff.h: rename DIFF_PLAIN color slot to DIFF_CONTEXT diff: accept color.diff.context as a synonym for "plain"
2015-06-11Merge branch 'jc/diff-ws-error-highlight'Libravatar Junio C Hamano1-22/+100
Allow whitespace breakages in deleted and context lines to be also painted in the output. * jc/diff-ws-error-highlight: diff.c: --ws-error-highlight=<kind> option diff.c: add emit_del_line() and emit_context_line() t4015: separate common setup and per-test expectation t4015: modernise style
2015-05-27diff.h: rename DIFF_PLAIN color slot to DIFF_CONTEXTLibravatar Jeff King1-13/+13
The latter is a much more descriptive name (and we support "color.diff.context" now). This also updates the name of any local variables which were used to store the color. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-27diff: accept color.diff.context as a synonym for "plain"Libravatar Jeff King1-1/+1
The term "plain" is a bit ambiguous; let's allow the more specific "context", but keep "plain" around for compatibility. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-26diff.c: --ws-error-highlight=<kind> optionLibravatar Junio C Hamano1-16/+68
Traditionally, we only cared about whitespace breakages introduced in new lines. Some people want to paint whitespace breakages on old lines, too. When they see a whitespace breakage on a new line, they can spot the same kind of whitespace breakage on the corresponding old line and want to say "Ah, those breakages are there but they were inherited from the original, so let's not touch them for now." Introduce `--ws-error-highlight=<kind>` option, that lets them pass a comma separated list of `old`, `new`, and `context` to specify what lines to highlight whitespace errors on. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-26diff.c: add emit_del_line() and emit_context_line()Libravatar Junio C Hamano1-12/+38
Traditionally, we only had emit_add_line() helper, which knows how to find and paint whitespace breakages on the given line, because we only care about whitespace breakages introduced in new lines. The context lines and old (i.e. deleted) lines are emitted with a simpler emit_line_0() that paints the entire line in plain or old colors. Identify callers of emit_line_0() that show deleted lines and context lines, have them call new helpers, emit_del_line() and emit_context_line(), so that we can later tweak what is done to these two classes of lines. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-17Merge branch 'rs/deflate-init-cleanup'Libravatar Junio C Hamano1-1/+0
Code simplification. * rs/deflate-init-cleanup: zlib: initialize git_zstream in git_deflate_init{,_gzip,_raw}
2015-03-06Merge branch 'mk/diff-shortstat-dirstat-fix'Libravatar Junio C Hamano1-1/+1
"git diff --shortstat --dirstat=changes" showed a dirstat based on lines that was never asked by the end user in addition to the dirstat that the user asked for. * mk/diff-shortstat-dirstat-fix: diff --shortstat --dirstat: remove duplicate output
2015-03-05zlib: initialize git_zstream in git_deflate_init{,_gzip,_raw}Libravatar René Scharfe1-1/+0
Clear the git_zstream variable at the start of git_deflate_init() etc. so that callers don't have to do that. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-02diff --shortstat --dirstat: remove duplicate outputLibravatar Mårten Kongstad1-1/+1
When --shortstat is used in conjunction with --dirstat=changes, git diff will output the dirstat information twice: first as calculated by the 'lines' algorithm, then as calculated by the 'changes' algorithm: $ git diff --dirstat=changes,10 --shortstat v2.2.0..v2.2.1 23 files changed, 453 insertions(+), 54 deletions(-) 33.5% Documentation/RelNotes/ 26.2% t/ 46.6% Documentation/RelNotes/ 16.6% t/ The same duplication happens for --shortstat together with --dirstat=files, but not for --shortstat together with --dirstat=lines. Limit output to only include one dirstat part, calculated as specified by the --dirstat parameter. Also, add test for this. Signed-off-by: Mårten Kongstad <marten.kongstad@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-20Merge branch 'jn/parse-config-slot'Libravatar Junio C Hamano1-2/+1
Code cleanup. * jn/parse-config-slot: color_parse: do not mention variable name in error message pass config slots as pointers instead of offsets
2014-10-14color_parse: do not mention variable name in error messageLibravatar Jeff King1-2/+1
Originally the color-parsing function was used only for config variables. It made sense to pass the variable name so that the die() message could be something like: $ git -c color.branch.plain=bogus branch fatal: bad color value 'bogus' for variable 'color.branch.plain' These days we call it in other contexts, and the resulting error messages are a little confusing: $ git log --pretty='%C(bogus)' fatal: bad color value 'bogus' for variable '--pretty format' $ git config --get-color foo.bar bogus fatal: bad color value 'bogus' for variable 'command line' This patch teaches color_parse to complain only about the value, and then return an error code. Config callers can then propagate that up to the config parser, which mentions the variable name. Other callers can provide a custom message. After this patch these three cases now look like: $ git -c color.branch.plain=bogus branch error: invalid color value: bogus fatal: unable to parse 'color.branch.plain' from command-line config $ git log --pretty='%C(bogus)' error: invalid color value: bogus fatal: unable to parse --pretty format $ git config --get-color foo.bar bogus error: invalid color value: bogus fatal: unable to parse default color value Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-11Merge branch 'nd/large-blobs'Libravatar Junio C Hamano1-14/+38
Teach a few codepaths to punt (instead of dying) when large blobs that would not fit in core are involved in the operation. * nd/large-blobs: diff: shortcut for diff'ing two binary SHA-1 objects diff --stat: mark any file larger than core.bigfilethreshold binary diff.c: allow to pass more flags to diff_populate_filespec sha1_file.c: do not die failing to malloc in unpack_compressed_entry wrapper.c: introduce gentle xmallocz that does not die()
2014-08-20run-command: introduce CHILD_PROCESS_INITLibravatar René Scharfe1-2/+1
Most struct child_process variables are cleared using memset first after declaration. Provide a macro, CHILD_PROCESS_INIT, that can be used to initialize them statically instead. That's shorter, doesn't require a function call and is slightly more readable (especially given that we already have STRBUF_INIT, ARGV_ARRAY_INIT etc.). Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-18diff: shortcut for diff'ing two binary SHA-1 objectsLibravatar Nguyễn Thái Ngọc Duy1-0/+13
If we are given two SHA-1 and asked to determine if they are different (but not _what_ differences), we know right away by comparing SHA-1. A side effect of this patch is, because large files are marked binary, diff-tree will not need to unpack them. 'diff-index --cached' will not either. But 'diff-files' still does. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-18diff --stat: mark any file larger than core.bigfilethreshold binaryLibravatar Nguyễn Thái Ngọc Duy1-8/+18
Too large files may lead to failure to allocate memory. If it happens here, it could impact quite a few commands that involve diff. Moreover, too large files are inefficient to compare anyway (and most likely non-text), so mark them binary and skip looking at their content. Noticed-by: Dale R. Worley <worley@alum.mit.edu> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-18diff.c: allow to pass more flags to diff_populate_filespecLibravatar Nguyễn Thái Ngọc Duy1-6/+7
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-22Merge branch 'bg/xcalloc-nmemb-then-size' into maintLibravatar Junio C Hamano1-1/+1
* bg/xcalloc-nmemb-then-size: transport-helper.c: rearrange xcalloc arguments remote.c: rearrange xcalloc arguments reflog-walk.c: rearrange xcalloc arguments pack-revindex.c: rearrange xcalloc arguments notes.c: rearrange xcalloc arguments imap-send.c: rearrange xcalloc arguments http-push.c: rearrange xcalloc arguments diff.c: rearrange xcalloc arguments config.c: rearrange xcalloc arguments commit.c: rearrange xcalloc arguments builtin/remote.c: rearrange xcalloc arguments builtin/ls-remote.c: rearrange xcalloc arguments
2014-07-17strbuf: use strbuf_addstr() for adding C stringsLibravatar René Scharfe1-6/+6
Avoid code duplication and let strbuf_addstr() call strlen() for us. Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-25Merge branch 'jk/diff-follow-must-take-one-pathspec' into maintLibravatar Junio C Hamano1-0/+3
"git format-patch" did not enforce the rule that the "--follow" option from the log/diff family of commands must be used with exactly one pathspec. * jk/diff-follow-must-take-one-pathspec: move "--follow needs one pathspec" rule to diff_setup_done
2014-06-20stat_opt: check extra strlen callLibravatar Jeff King1-1/+2
As in earlier commits, the diff option parser uses starts_with to find that an argument starts with "--stat-", and then adds strlen("stat-") to find the rest of the option. However, in this case the starts_with and the strlen are separated across functions, making it easy to call the latter without the former. Let's use skip_prefix instead of raw pointer arithmetic to catch such a case. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-20use skip_prefix to avoid repeating stringsLibravatar Jeff King1-18/+9
It's a common idiom to match a prefix and then skip past it with strlen, like: if (starts_with(foo, "bar")) foo += strlen("bar"); This avoids magic numbers, but means we have to repeat the string (and there is no compiler check that we didn't make a typo in one of the strings). We can use skip_prefix to handle this case without repeating ourselves. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-20use skip_prefix to avoid magic numbersLibravatar Jeff King1-31/+34
It's a common idiom to match a prefix and then skip past it with a magic number, like: if (starts_with(foo, "bar")) foo += 3; This is easy to get wrong, since you have to count the prefix string yourself, and there's no compiler check if the string changes. We can use skip_prefix to avoid the magic numbers here. Note that some of these conversions could be much shorter. For example: if (starts_with(arg, "--foo=")) { bar = arg + 6; continue; } could become: if (skip_prefix(arg, "--foo=", &bar)) continue; However, I have left it as: if (skip_prefix(arg, "--foo=", &v)) { bar = v; continue; } to visually match nearby cases which need to actually process the string. Like: if (skip_prefix(arg, "--foo=", &v)) { bar = atoi(v); continue; } Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-18parse_diff_color_slot: drop ofs parameterLibravatar Jeff King1-10/+10
This function originally took a whole config variable name ("var") and an offset ("ofs"). It checked "var+ofs" against each color slot, but reported errors using the whole "var". However, since 8b8e862 (ignore unknown color configuration, 2009-12-12), it returns -1 rather than printing its own error, and therefore only cares about var+ofs. We can drop the ofs parameter and teach its sole caller to derive the pointer itself. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Merge branch 'bg/xcalloc-nmemb-then-size'Libravatar Junio C Hamano1-1/+1
Like calloc(3), xcalloc() takes nmemb and then size. * bg/xcalloc-nmemb-then-size: transport-helper.c: rearrange xcalloc arguments remote.c: rearrange xcalloc arguments reflog-walk.c: rearrange xcalloc arguments pack-revindex.c: rearrange xcalloc arguments notes.c: rearrange xcalloc arguments imap-send.c: rearrange xcalloc arguments http-push.c: rearrange xcalloc arguments diff.c: rearrange xcalloc arguments config.c: rearrange xcalloc arguments commit.c: rearrange xcalloc arguments builtin/remote.c: rearrange xcalloc arguments builtin/ls-remote.c: rearrange xcalloc arguments
2014-06-16Merge branch 'jk/diff-follow-must-take-one-pathspec'Libravatar Junio C Hamano1-0/+3
* jk/diff-follow-must-take-one-pathspec: move "--follow needs one pathspec" rule to diff_setup_done
2014-06-03Merge branch 'jk/external-diff-use-argv-array'Libravatar Junio C Hamano1-31/+25
Code clean-up (and a bugfix which has been merged for 2.0). * jk/external-diff-use-argv-array: run_external_diff: refactor cmdline setup logic run_external_diff: hoist common bits out of conditional run_external_diff: drop fflush(NULL) run_external_diff: clean up error handling run_external_diff: use an argv_array for the environment
2014-06-03Merge branch 'ks/tree-diff-nway'Libravatar Junio C Hamano1-0/+2
Instead of running N pair-wise diff-trees when inspecting a N-parent merge, find the set of paths that were touched by walking N+1 trees in parallel. These set of paths can then be turned into N pair-wise diff-tree results to be processed through rename detections and such. And N=2 case nicely degenerates to the usual 2-way diff-tree, which is very nice. * ks/tree-diff-nway: mingw: activate alloca combine-diff: speed it up, by using multiparent diff tree-walker directly tree-diff: rework diff_tree() to generate diffs for multiparent cases as well Portable alloca for Git tree-diff: reuse base str(buf) memory on sub-tree recursion tree-diff: no need to call "full" diff_tree_sha1 from show_path() tree-diff: rework diff_tree interface to be sha1 based tree-diff: diff_tree() should now be static tree-diff: remove special-case diff-emitting code for empty-tree cases tree-diff: simplify tree_entry_pathcmp tree-diff: show_path prototype is not needed anymore tree-diff: rename compare_tree_entry -> tree_entry_pathcmp tree-diff: move all action-taking code out of compare_tree_entry() tree-diff: don't assume compare_tree_entry() returns -1,0,1 tree-diff: consolidate code for emitting diffs and recursion in one place tree-diff: show_tree() is not needed tree-diff: no need to pass match to skip_uninteresting() tree-diff: no need to manually verify that there is no mode change for a path combine-diff: move changed-paths scanning logic into its own function combine-diff: move show_log_first logic/action out of paths scanning
2014-05-27diff.c: rearrange xcalloc argumentsLibravatar Brian Gesiak1-1/+1
xcalloc() takes two arguments: the number of elements and their size. diffstat_add() passes the arguments in reverse order, passing the size of a diffstat_file*, followed by the number of diffstat_file* to be allocated. Rearrange them so they are in the correct order. Signed-off-by: Brian Gesiak <modocache@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-20move "--follow needs one pathspec" rule to diff_setup_doneLibravatar Jeff King1-0/+3
Because of the way "--follow" is implemented, we must have exactly one pathspec. "git log" enforces this restriction, but other users of the revision traversal code do not. For example, "git format-patch --follow" will segfault during try_to_follow_renames, as we have no pathspecs at all. We can push this check down into diff_setup_done, which is probably a better place anyway. It is the diff code that introduces this restriction, so other parts of the code should not need to care themselves. Reported-by: "Michael S. Tsirkin" <mst@redhat.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-28Merge branch 'jk/external-diff-use-argv-array' (early part)Libravatar Junio C Hamano1-16/+16
Crash fix for codepath that miscounted the necessary size for an array when spawning an external diff program. * 'jk/external-diff-use-argv-array' (early part): run_external_diff: use an argv_array for the command line
2014-04-21run_external_diff: refactor cmdline setup logicLibravatar Jeff King1-11/+15
The current logic makes it hard to see what gets put onto the command line in which cases. Pulling out a helper function lets us see that we have two sets of file data, and the second set either uses the original name, or the "other" renamed/copy name. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21run_external_diff: hoist common bits out of conditionalLibravatar Jeff King1-5/+3
Whether we have diff_filespecs to give to the diff command or not, we always are going to run the program and pass it the pathname. Let's pull that duplicated part out of the conditional to make it more obvious. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21run_external_diff: drop fflush(NULL)Libravatar Jeff King1-1/+0
This fflush was added in d5535ec (Use run_command() to spawn external diff programs instead of fork/exec., 2007-10-19), because flushing buffers before forking is a good habit. But later, 7d0b18a (Add output flushing before fork(), 2008-08-04) added it to the generic run-command interface, meaning that our flush here is redundant. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21run_external_diff: clean up error handlingLibravatar Jeff King1-6/+3
When the external diff reports an error, we try to clean up and die. However, we can make this process a bit simpler: 1. We do not need to bother freeing memory, since we are about to exit. Nor do we need to clean up our tempfiles, since the atexit() handler will do it for us. So we can die as soon as we see the error. 3. We can just call die() rather than fprintf/exit. This does technically change our exit code, but the exit code of "1" is not meaningful here. In fact, it is probably wrong, since "1" from diff usually means "completed successfully, but there were differences". And while we're there, we can mark the error message for translation, and drop the full stop at the end to make it more like our other messages. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21run_external_diff: use an argv_array for the environmentLibravatar Jeff King1-9/+5
We currently use static buffers and a static array for formatting the environment passed to the external diff. There's nothing wrong in the code, but it is much easier to verify that it is correct if we use a dynamic argv_array. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21run_external_diff: use an argv_array for the command lineLibravatar Jeff King1-16/+16
We currently generate the command-line for the external command using a fixed-length array of size 10. But if there is a rename, we actually need 11 elements (10 items, plus a NULL), and end up writing a random NULL onto the stack. Rather than bump the limit, let's just use an argv_array, which makes this sort of error impossible. Noticed-by: Max L <infthi.inbox@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-17i18n: remove obsolete comments for translators in diffstat generationLibravatar Jiang Xin1-8/+0
Since we do not translate diffstat any more, remove the obsolete comments. Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-08Merge branch 'jl/nor-or-nand-and'Libravatar Junio C Hamano1-1/+1
Eradicate mistaken use of "nor" (that is, essentially "nor" used not in "neither A nor B" ;-)) from in-code comments, command output strings, and documentations. * jl/nor-or-nand-and: code and test: fix misuses of "nor" comments: fix misuses of "nor" contrib: fix misuses of "nor" Documentation: fix misuses of "nor"
2014-04-07combine-diff: speed it up, by using multiparent diff tree-walker directlyLibravatar Kirill Smelkov1-0/+1
As was recently shown in "combine-diff: optimize combine_diff_path sets intersection", combine-diff runs very slowly. In that commit we optimized paths sets intersection, but that accounted only for ~ 25% of the slowness, and as my tracing showed, for linux.git v3.10..v3.11, for merges a lot of time is spent computing diff(commit,commit^2) just to only then intersect that huge diff to almost small set of files from diff(commit,commit^1). In previous commit, we described the problem in more details, and reworked the diff tree-walker to be general one - i.e. to work in multiple parent case too. Now is the time to take advantage of it for finding paths for combine diff. The implementation is straightforward - if we know, we can get generated diff paths directly, and at present that means no diff filtering or rename/copy detection was requested(*), we can call multiparent tree-walker directly and get ready paths. (*) because e.g. at present, all diffcore transformations work on diff_filepair queues, but in the future, that limitation can be lifted, if filters would operate directly on combine_diff_paths. Timings for `git log --raw --no-abbrev --no-renames` without `-c` ("git log") and with `-c` ("git log -c") and with `-c --merges` ("git log -c --merges") before and after the patch are as follows: linux.git v3.10..v3.11 log log -c log -c --merges before 1.9s 16.4s 15.2s after 1.9s 2.4s 1.1s The result stayed the same. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07tree-diff: rework diff_tree() to generate diffs for multiparent cases as wellLibravatar Kirill Smelkov1-0/+1
Previously diff_tree(), which is now named ll_diff_tree_sha1(), was generating diff_filepair(s) for two trees t1 and t2, and that was usually used for a commit as t1=HEAD~, and t2=HEAD - i.e. to see changes a commit introduces. In Git, however, we have fundamentally built flexibility in that a commit can have many parents - 1 for a plain commit, 2 for a simple merge, but also more than 2 for merging several heads at once. For merges there is a so called combine-diff, which shows diff, a merge introduces by itself, omitting changes done by any parent. That works through first finding paths, that are different to all parents, and then showing generalized diff, with separate columns for +/- for each parent. The code lives in combine-diff.c . There is an impedance mismatch, however, in that a commit could generally have any number of parents, and that while diffing trees, we divide cases for 2-tree diffs and more-than-2-tree diffs. I mean there is no special casing for multiple parents commits in e.g. revision-walker . That impedance mismatch *hurts* *performance* *badly* for generating combined diffs - in "combine-diff: optimize combine_diff_path sets intersection" I've already removed some slowness from it, but from the timings provided there, it could be seen, that combined diffs still cost more than an order of magnitude more cpu time, compared to diff for usual commits, and that would only be an optimistic estimate, if we take into account that for e.g. linux.git there is only one merge for several dozens of plain commits. That slowness comes from the fact that currently, while generating combined diff, a lot of time is spent computing diff(commit,commit^2) just to only then intersect that huge diff to almost small set of files from diff(commit,commit^1). That's because at present, to compute combine-diff, for first finding paths, that "every parent touches", we use the following combine-diff property/definition: D(A,P1...Pn) = D(A,P1) ^ ... ^ D(A,Pn) (w.r.t. paths) where D(A,P1...Pn) is combined diff between commit A, and parents Pi and D(A,Pi) is usual two-tree diff Pi..A So if any of that D(A,Pi) is huge, tracting 1 n-parent combine-diff as n 1-parent diffs and intersecting results will be slow. And usually, for linux.git and other topic-based workflows, that D(A,P2) is huge, because, if merge-base of A and P2, is several dozens of merges (from A, via first parent) below, that D(A,P2) will be diffing sum of merges from several subsystems to 1 subsystem. The solution is to avoid computing n 1-parent diffs, and to find changed-to-all-parents paths via scanning A's and all Pi's trees simultaneously, at each step comparing their entries, and based on that comparison, populate paths result, and deduce we could *skip* *recursing* into subdirectories, if at least for 1 parent, sha1 of that dir tree is the same as in A. That would save us from doing significant amount of needless work. Such approach is very similar to what diff_tree() does, only there we deal with scanning only 2 trees simultaneously, and for n+1 tree, the logic is a bit more complex: D(T,P1...Pn) calculation scheme ------------------------------- D(T,P1...Pn) = D(T,P1) ^ ... ^ D(T,Pn) (regarding resulting paths set) D(T,Pj) - diff between T..Pj D(T,P1...Pn) - combined diff from T to parents P1,...,Pn We start from all trees, which are sorted, and compare their entries in lock-step: T P1 Pn - - - |t| |p1| |pn| |-| |--| ... |--| imin = argmin(p1...pn) | | | | | | |-| |--| |--| |.| |. | |. | . . . . . . at any time there could be 3 cases: 1) t < p[imin]; 2) t > p[imin]; 3) t = p[imin]. Schematic deduction of what every case means, and what to do, follows: 1) t < p[imin] -> ∀j t ∉ Pj -> "+t" ∈ D(T,Pj) -> D += "+t"; t↓ 2) t > p[imin] 2.1) ∃j: pj > p[imin] -> "-p[imin]" ∉ D(T,Pj) -> D += ø; ∀ pi=p[imin] pi↓ 2.2) ∀i pi = p[imin] -> pi ∉ T -> "-pi" ∈ D(T,Pi) -> D += "-p[imin]"; ∀i pi↓ 3) t = p[imin] 3.1) ∃j: pj > p[imin] -> "+t" ∈ D(T,Pj) -> only pi=p[imin] remains to investigate 3.2) pi = p[imin] -> investigate δ(t,pi) | | v 3.1+3.2) looking at δ(t,pi) ∀i: pi=p[imin] - if all != ø -> ⎧δ(t,pi) - if pi=p[imin] -> D += ⎨ ⎩"+t" - if pi>p[imin] in any case t↓ ∀ pi=p[imin] pi↓ ~ For comparison, here is how diff_tree() works: D(A,B) calculation scheme ------------------------- A B - - |a| |b| a < b -> a ∉ B -> D(A,B) += +a a↓ |-| |-| a > b -> b ∉ A -> D(A,B) += -b b↓ | | | | a = b -> investigate δ(a,b) a↓ b↓ |-| |-| |.| |.| . . . . ~~~~~~~~ This patch generalizes diff tree-walker to work with arbitrary number of parents as described above - i.e. now there is a resulting tree t, and some parents trees tp[i] i=[0..nparent). The generalization builds on the fact that usual diff D(A,B) is by definition the same as combined diff D(A,[B]), so if we could rework the code for common case and make it be not slower for nparent=1 case, usual diff(t1,t2) generation will not be slower, and multiparent diff tree-walker would greatly benefit generating combine-diff. What we do is as follows: 1) diff tree-walker ll_diff_tree_sha1() is internally reworked to be a paths generator (new name diff_tree_paths()), with each generated path being `struct combine_diff_path` with info for path, new sha1,mode and for every parent which sha1,mode it was in it. 2) From that info, we can still generate usual diff queue with struct diff_filepairs, via "exporting" generated combine_diff_path, if we know we run for nparent=1 case. (see emit_diff() which is now named emit_diff_first_parent_only()) 3) In order for diff_can_quit_early(), which checks DIFF_OPT_TST(opt, HAS_CHANGES)) to work, that exporting have to be happening not in bulk, but incrementally, one diff path at a time. For such consumers, there is a new callback in diff_options introduced: ->pathchange(opt, struct combine_diff_path *) which, if set to !NULL, is called for every generated path. (see new compat ll_diff_tree_sha1() wrapper around new paths generator for setup) 4) The paths generation itself, is reworked from previous ll_diff_tree_sha1() code according to "D(A,P1...Pn) calculation scheme" provided above: On the start we allocate [nparent] arrays in place what was earlier just for one parent tree. then we just generalize loops, and comparison according to the algorithm. Some notes(*): 1) alloca(), for small arrays, is used for "runs not slower for nparent=1 case than before" goal - if we change it to xmalloc()/free() the timings get ~1% worse. For alloca() we use just-introduced xalloca/xalloca_free compatibility wrappers, so it should not be a portability problem. 2) For every parent tree, we need to keep a tag, whether entry from that parent equals to entry from minimal parent. For performance reasons I'm keeping that tag in entry's mode field in unused bit - see S_IFXMIN_NEQ. Not doing so, we'd need to alloca another [nparent] array, which hurts performance. 3) For emitted paths, memory could be reused, if we know the path was processed via callback and will not be needed later. We use efficient hand-made realloc-style path_appendnew(), that saves us from ~1-1.5% of potential additional slowdown. 4) goto(s) are used in several places, as the code executes a little bit faster with lowered register pressure. Also - we should now check for FIND_COPIES_HARDER not only when two entries names are the same, and their hashes are equal, but also for a case, when a path was removed from some of all parents having it. The reason is, if we don't, that path won't be emitted at all (see "a > xi" case), and we'll just skip it, and FIND_COPIES_HARDER wants all paths - with diff or without - to be emitted, to be later analyzed for being copies sources. The new check is only necessary for nparent >1, as for nparent=1 case xmin_eqtotal always =1 =nparent, and a path is always added to diff as removal. ~~~~~~~~ Timings for # without -c, i.e. testing only nparent=1 case `git log --raw --no-abbrev --no-renames` before and after the patch are as follows: navy.git linux.git v3.10..v3.11 before 0.611s 1.889s after 0.619s 1.907s slowdown 1.3% 0.9% This timings show we did no harm to usual diff(tree1,tree2) generation. From the table we can see that we actually did ~1% slowdown, but I think I've "earned" that 1% in the previous patch ("tree-diff: reuse base str(buf) memory on sub-tree recursion", HEAD~~) so for nparent=1 case, net timings stays approximately the same. The output also stayed the same. (*) If we revert 1)-4) to more usual techniques, for nparent=1 case, we'll get ~2-2.5% of additional slowdown, which I've tried to avoid, as "do no harm for nparent=1 case" rule. For linux.git, combined diff will run an order of magnitude faster and appropriate timings will be provided in the next commit, as we'll be taking advantage of the new diff tree-walker for combined-diff generation there. P.S. and combined diff is not some exotic/for-play-only stuff - for example for a program I write to represent Git archives as readonly filesystem, there is initial scan with `git log --reverse --raw --no-abbrev --no-renames -c` to extract log of what was created/changed when, as a result building a map {} sha1 -> in which commit (and date) a content was added that `-c` means also show combined diff for merges, and without them, if a merge is non-trivial (merges changes from two parents with both having separate changes to a file), or an evil one, the map will not be full, i.e. some valid sha1 would be absent from it. That case was my initial motivation for combined diffs speedup. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-31comments: fix misuses of "nor"Libravatar Justin Lebar1-1/+1
Signed-off-by: Justin Lebar <jlebar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>