summaryrefslogtreecommitdiff
path: root/builtin/fast-export.c
AgeCommit message (Collapse)AuthorFilesLines
2013-05-07fast-{import,export}: use get_sha1_hex() to read from marks fileLibravatar Felipe Contreras1-1/+1
It's wrong to call get_sha1() if they should be SHA-1s, plus inefficient. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-07fast-export: don't parse commits while reading marks fileLibravatar Felipe Contreras1-1/+6
We don't need the parsed objects at this point, merely the information that they have marks. Seems to be three times faster in my setup with lots of objects. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-07fast-export: do not parse non-commit objects while reading marks fileLibravatar Felipe Contreras1-6/+9
We read from the marks file and keep only marked commits, but in order to find the type of object, we are parsing the whole thing, which is slow, specially in big repositories with lots of big files. There's no need for that, we can query the object information with sha1_object_info(). Before this, loading the objects of a fresh emacs import, with 260598 blobs took 14 minutes, after this patch, it takes 3 seconds. This is the way fast-import does it. Also die if the object is not found (like fast-import). Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-15fast-export: add --signed-tags=warn-strip modeLibravatar John Keeping1-1/+7
This issues a warning while stripping signatures from signed tags, which allows us to use it as default behaviour for remote helpers which cannot specify how to handle signed tags. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-12Sync with 'maint'Libravatar Junio C Hamano1-2/+2
* maint: Correct common spelling mistakes in comments and tests kwset: fix spelling in comments precompose-utf8: fix spelling of "want" in error message compat/nedmalloc: fix spelling in comments compat/regex: fix spelling and grammar in comments obstack: fix spelling of similar contrib/subtree: fix spelling of accidentally git-remote-mediawiki: spelling fixes doc: various spelling fixes fast-export: fix argument name in error messages Documentation: distinguish between ref and offset deltas in pack-format i18n: make the translation of -u advice in one go
2013-04-12Merge branch 'maint-1.8.1' into maintLibravatar Junio C Hamano1-2/+2
* maint-1.8.1: fast-export: fix argument name in error messages Documentation: distinguish between ref and offset deltas in pack-format
2013-04-12fast-export: fix argument name in error messagesLibravatar Paul Price1-2/+2
The --signed-tags argument is plural, while error messages referred to --signed-tag (singular). Tweak error messages to correspond to the argument. Signed-off-by: Paul Price <price@astro.princeton.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-07fast-export: Allow pruned-references in mark fileLibravatar Antoine Pelisse1-4/+7
fast-export can fail because of some pruned-reference when importing a mark file. The problem happens in the following scenario: $ git fast-export --export-marks=MARKS master (rewrite master) $ git prune $ git fast-export --import-marks=MARKS master This might fail if some references have been removed by prune because some marks will refer to no longer existing commits. git-fast-export will not need these objects anyway as they were no longer reachable. We still need to update last_numid so we don't change the mapping between marks and objects for remote-helpers. Unfortunately, the mark file should not be rewritten without lost marks if no new objects has been exported, as we could lose track of the last last_numid. Signed-off-by: Antoine Pelisse <apelisse@gmail.com> Reviewed-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-25Merge branch 'jk/fast-export-object-lookup'Libravatar Junio C Hamano1-9/+13
* jk/fast-export-object-lookup: fast-export: do not load blob objects twice fast-export: rename handle_object function
2013-03-17fast-export: do not load blob objects twiceLibravatar Jeff King1-6/+10
When fast-export wants to export a blob object, it first calls parse_object to get a "struct object" and check whether we have already shown the object. If we haven't shown it, we then use read_sha1_file to pull it from disk and write it out. That means we load each blob from disk twice: once for parse_object to find its type and check its sha1, and a second time when we actually output it. We can drop this to a single load by using lookup_object to check the SHOWN flag, and then checking the signature on and outputting a single buffer. This provides modest speedups on git.git (best-of-five, "git fast-export HEAD >/dev/null"): [before] [after] real 0m14.347s real 0m13.780s user 0m14.084s user 0m13.620s sys 0m0.208s sys 0m0.100s and somewhat more on more blob-heavy repos (this is a repository full of media files): [before] [after] real 0m52.236s real 0m44.451s user 0m50.568s user 0m43.000s sys 0m1.536s sys 0m1.284s Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-17fast-export: rename handle_object functionLibravatar Jeff King1-3/+3
The handle_object function is rather vaguely named; it only operates on blobs, and its purpose is to export the blob to the output stream. Let's call it "export_blob" to make it more clear what it does. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-03fast-export: make sure updated refs get updatedLibravatar Felipe Contreras1-3/+7
When an object has already been exported (and thus is in the marks) it's flagged as SHOWN, so it will not be exported again, even if in a later time it's exported through a different ref. We don't need the object to be exported again, but we want the ref updated, which doesn't happen. Since we can't know if a ref was exported or not, let's just assume that if the commit was marked (flags & SHOWN), the user still wants the ref updated. IOW: If it's specified in the command line, it will get updated, regardless of whether or not the object was marked. So: % git branch test master % git fast-export $mark_flags master % git fast-export $mark_flags test Would export 'test' properly. Additionally, this fixes issues with remote helpers; now they can push refs whose objects have already been exported, and a few other issues as well. Update the tests accordingly. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-03fast-export: don't handle uninteresting refsLibravatar Felipe Contreras1-4/+7
They have been marked as UNINTERESTING for a reason, lets respect that. Currently the first ref is handled properly, but not the rest. Assuming that all the refs point at the same commit in the following example: % git fast-export master ^uninteresting ^foo ^bar reset refs/heads/bar from :0 reset refs/heads/foo from :0 reset refs/heads/uninteresting from :0 % git fast-export ^uninteresting ^foo ^bar master reset refs/heads/master from :0 reset refs/heads/bar from :0 reset refs/heads/foo from :0 Clearly this is wrong; the negative refs should be ignored. After this patch: % git fast-export ^uninteresting ^foo ^bar master # nothing % git fast-export master ^uninteresting ^foo ^bar # nothing And even more, it would only happen if the ref is pointing to exactly the same commit, but not otherwise: % git fast-export ^next next reset refs/heads/next from :0 % git fast-export ^next next^{commit} # nothing % git fast-export ^next next~0 # nothing % git fast-export ^next next~1 # nothing % git fast-export ^next next~2 # nothing The reason this happens is that before traversing the commits, fast-export checks if any of the refs point to the same object, and any duplicated ref gets added to a list in order to issue 'reset' commands after the traversing. Unfortunately, it's not even checking if the commit is flagged as UNINTERESTING. The fix of course, is to check it. However, in order to do it properly we need to get the UNINTERESTING flag from the command line, not from the commit object, because "^foo bar" will mark the commit 'bar' uninteresting if foo and bar points at the same commit. rev_cmdline_info, which was introduced exactly to handle this situation, contains all the information we need for get_tags_and_duplicates(), plus the ref flag. This way the rest of the positive refs will remain untouched; it's only the negative ones that change in behavior. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-29fast-export: trivial cleanupLibravatar Felipe Contreras1-1/+1
Setting 'commit' to 'commit' is a no-op. It might have been there to avoid a compiler warning, but if so, it was the compiler to blame, and it's certainly not there any more. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-26fast-export: avoid importing blob marksLibravatar Felipe Contreras1-0/+4
We want to be able to import, and then export, using the same marks, so that we don't push things that the other side already received. Unfortunately, fast-export doesn't store blobs in the marks, but fast-import does. This creates a mismatch when fast export is reusing a mark that was previously stored by fast-import. There is no point in one tool saving blobs, and the other not, but for now let's just check in fast-export that the objects are indeed commits. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-20i18n: fast-export: mark parseopt strings for translationLibravatar Nguyễn Thái Ngọc Duy1-14/+14
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-09Merge branch 'js/fast-export-paths-with-spaces'Libravatar Junio C Hamano1-0/+2
"git fast-export" produced an input stream for fast-import without properly quoting pathnames when they contain SPs in them. * js/fast-export-paths-with-spaces: fast-export: quote paths with spaces
2012-06-27fast-export: quote paths with spacesLibravatar Jay Soffian1-0/+2
A path containing a space must be quoted when used as an argument to either the copy or rename commands (because unlike other commands, the path is not the final thing on the line for those commands). Commit 6280dfdc3b (fast-export: quote paths in output, 2011-08-05) previously attempted to fix fast-export's quoting by passing all paths through quote_c_style(). However, that function does not consider the space to be a character which requires quoting, so let's special-case the space inside print_path(). This will cause space-containing paths to also be quoted in other commands where such quoting is not strictly necessary, but it does not hurt to do so. The test from 6280dfdc3b did not detect this because, while it does introduce renames in the export stream, it does not actually turn on rename detection, so they were presented as pairs of deletions/adds. Using "-M" reveals the bug. Signed-off-by: Jay Soffian <jaysoffian@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-12Merge branch 'js/maint-fast-export-mark-error'Libravatar Junio C Hamano1-1/+1
* js/maint-fast-export-mark-error: fast-export: report SHA-1 instead of gibberish when marks exist already
2012-03-01Merge branch 'rs/no-no-no-parseopt'Libravatar Junio C Hamano1-3/+1
* rs/no-no-no-parseopt: parse-options: remove PARSE_OPT_NEGHELP parse-options: allow positivation of options starting, with no- test-parse-options: convert to OPT_BOOL() Conflicts: builtin/grep.c
2012-02-28parse-options: remove PARSE_OPT_NEGHELPLibravatar René Scharfe1-3/+1
PARSE_OPT_NEGHELP is confusing because short options defined with that flag do the opposite of what the helptext says. It is also not needed anymore now that options starting with no- can be negated by removing that prefix. Convert its only two users to OPT_NEGBIT() and OPT_BOOL() and then remove support for PARSE_OPT_NEGHELP. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-21Fix an enum assignment issue spotted by Sun StudioLibravatar Ævar Arnfjörð Bjarmason1-2/+2
In builtin/fast-export.c we'd assign to variables of the tag_of_filtered_mode enum type with constants defined for the signed_tag_mode enum. We'd get the intended value since both the value we were assigning with and the one we actually wanted had the same positional within their respective enums, but doing it this way makes no sense. This issue was spotted by Sun Studio 12 Update 1: "builtin/fast-export.c", line 54: warning: enum type mismatch: op "=" (E_ENUM_TYPE_MISMATCH_OP) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-11Merge branch 'jk/fast-export-quote-path'Libravatar Junio C Hamano1-7/+24
* jk/fast-export-quote-path: fast-export: quote paths in output
2011-08-05fast-export: quote paths in outputLibravatar Jeff King1-7/+24
Many pathnames in a fast-import stream need to be quoted. In particular: 1. Pathnames at the end of an "M" or "D" line need quoting if they contain a LF or start with double-quote. 2. Pathnames on a "C" or "R" line need quoting as above, but also if they contain spaces. For (1), we weren't quoting at all. For (2), we put double-quotes around the paths to handle spaces, but ignored the possibility that they would need further quoting. This patch checks whether each pathname needs c-style quoting, and uses it. This is slightly overkill for (1), which doesn't actually need to quote many characters that vanilla c-style quoting does. However, it shouldn't hurt, as any implementation needs to be ready to handle quoted strings anyway. In addition to adding a test, we have to tweak a test which blindly assumed that case (2) would always use double-quotes, whether it needed to or not. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-07-19fast-export: support done featureLibravatar Sverre Rabbelier1-0/+9
If fast-export is being used to generate a fast-import stream that will be used afterwards it is desirable to indicate the end of the stream with the new 'done' command. Add a flag that causes fast-export to end with 'done'. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-27Merge branch 'mg/placeholders-are-lowercase'Libravatar Junio C Hamano1-2/+2
* mg/placeholders-are-lowercase: Make <identifier> lowercase in Documentation Make <identifier> lowercase as per CodingGuidelines Make <identifier> lowercase as per CodingGuidelines Make <identifier> lowercase as per CodingGuidelines CodingGuidelines: downcase placeholders in usage messages
2011-02-15Make <identifier> lowercase as per CodingGuidelinesLibravatar Michael J Gruber1-2/+2
*.c part for matches with '"[A-Z]+"'. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-03struct rev_info: convert prune_data to struct pathspecLibravatar Nguyễn Thái Ngọc Duy1-1/+1
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-09-08Merge branch 'en/d-f-conflict-fix'Libravatar Junio C Hamano1-1/+9
* en/d-f-conflict-fix: fast-export: ensure that a renamed file is printed after all references
2010-09-08fast-export: ensure that a renamed file is printed after all referencesLibravatar Johannes Sixt1-1/+9
t9350 sets up a commit where a file is both copied and renamed. The output of fast-export for this commit should look like this: author ... committer ... from :19 C "file2" "file4" R "file2" "file5" The order of the two modification lines is derived from the result that the diff machinery produces. 060df62 (fast-export: Fix output order of D/F changes) inserted a qsort call that modifies the order of the diff result. Unfortunately, qsort need not be stable. Therefore, it is possible that the 'R' line appears before the 'C' line and the resulting fast-import stream is incorrect. Fix it by forcing that the rename entry is printed after all other modification lines with the same file name. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31Merge branch 'en/d-f-conflict-fix'Libravatar Junio C Hamano1-0/+29
* en/d-f-conflict-fix: merge-recursive: Avoid excessive output for and reprocessing of renames merge-recursive: Fix multiple file rename across D/F conflict t6031: Add a testcase covering multiple renames across a D/F conflict merge-recursive: Fix typo Mark tests that use symlinks as needing SYMLINKS prerequisite t/t6035-merge-dir-to-symlink.sh: Remove TODO on passing test fast-import: Improve robustness when D->F changes provided in wrong order fast-export: Fix output order of D/F changes merge_recursive: Fix renames across paths below D/F conflicts merge-recursive: Fix D/F conflicts Add a rename + D/F conflict testcase Add additional testcases for D/F conflicts Conflicts: merge-recursive.c
2010-08-31Merge branch 'en/fast-export-fix'Libravatar Junio C Hamano1-1/+10
* en/fast-export-fix: fast-export: Add a --full-tree option fast-export: Fix dropping of files with --import-marks and path limiting
2010-08-29object.h: Add OBJECT_ARRAY_INIT macro and make use of it.Libravatar Thiago Farina1-1/+1
Signed-off-by: Thiago Farina <tfransosi@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-19fast-export: Add a --full-tree optionLibravatar Elijah Newren1-1/+3
This option adds symmetry with fast-import, enabling it to also work with complete trees instead of just incremental changes. It works by issuing a 'deleteall' directive with each commit and then listing the full set of files that make up that commit, rather than just showing the list of files that have changed since the (first) parent commit. Note that this functionality is automatically turned on when using --import-marks together with path limiting in order to avoid dropping important but unchanged files. This functionality is desired when using hand-written filters along with 'fast-export | some-filter | fast-import' as it can be easier to write <some-filter> in terms of complete trees than incremental changes. We could avoid the need to add this option by simply always turning it on. While the end result would be identical, it would slow things down slightly by printing many more filenames per commit which goes somewhat against the 'fast' in 'fast-export'. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-19fast-export: Fix dropping of files with --import-marks and path limitingLibravatar Elijah Newren1-1/+8
Since fast-export operates by listing file changes since the (first) parent commit, when using --import-marks and path limiting and using a wider list of paths than in previous runs, files from the new path(s) will silently be omitted from the result unless or until a commit which explicitly changes those files. The resulting repository in such cases is broken and makes no sense. This commit fixes this by having fast-export work with complete trees instead of incremental changes (when both --import-marks and path limiting are used). It works by issuing a 'deleteall' directive with each commit and then listing the full set of files that make up that commit, rather than just showing the list of files that have changed since the (first) parent commit. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-09fast-export: Fix output order of D/F changesLibravatar Elijah Newren1-0/+29
The fast-import stream format requires incremental changes which take place immediately, meaning that for D->F conversions all files below the relevant directory must be deleted before the resulting file of the same name is created. Reversing the order can result in fast-import silently deleting the file right after creating it, resulting in the file missing from the resulting repository. We correct this by first sorting the diff_queue_struct in depth-first order. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05string_list: Add STRING_LIST_INIT macro and make use of it.Libravatar Thiago Farina1-1/+1
Acked-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Thiago Farina <tfransosi@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-27string_list: Fix argument order for string_list_appendLibravatar Julian Phillips1-2/+2
Update the definition and callers of string_list_append to use the string_list as the first argument. This helps make the string_list API easier to use by being more consistent. Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-28Merge branch 'maint'Libravatar Junio C Hamano1-1/+1
* maint: t9350: fix careless use of "cd" difftool: Fix '--gui' when diff.guitool is unconfigured fast-export: don't segfault when marks file cannot be opened
2010-02-22Move 'builtin-*' into a 'builtin/' subdirectoryLibravatar Linus Torvalds1-0/+633
This shrinks the top-level directory a bit, and makes it much more pleasant to use auto-completion on the thing. Instead of [torvalds@nehalem git]$ em buil<tab> Display all 180 possibilities? (y or n) [torvalds@nehalem git]$ em builtin-sh builtin-shortlog.c builtin-show-branch.c builtin-show-ref.c builtin-shortlog.o builtin-show-branch.o builtin-show-ref.o [torvalds@nehalem git]$ em builtin-shor<tab> builtin-shortlog.c builtin-shortlog.o [torvalds@nehalem git]$ em builtin-shortlog.c you get [torvalds@nehalem git]$ em buil<tab> [type] builtin/ builtin.h [torvalds@nehalem git]$ em builtin [auto-completes to] [torvalds@nehalem git]$ em builtin/sh<tab> [type] shortlog.c shortlog.o show-branch.c show-branch.o show-ref.c show-ref.o [torvalds@nehalem git]$ em builtin/sho [auto-completes to] [torvalds@nehalem git]$ em builtin/shor<tab> [type] shortlog.c shortlog.o [torvalds@nehalem git]$ em builtin/shortlog.c which doesn't seem all that different, but not having that annoying break in "Display all 180 possibilities?" is quite a relief. NOTE! If you do this in a clean tree (no object files etc), or using an editor that has auto-completion rules that ignores '*.o' files, you won't see that annoying 'Display all 180 possibilities?' message - it will just show the choices instead. I think bash has some cut-off around 100 choices or something. So the reason I see this is that I'm using an odd editory, and thus don't have the rules to cut down on auto-completion. But you can simulate that by using 'ls' instead, or something similar. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>