summaryrefslogtreecommitdiff
path: root/t/t9350-fast-export.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-11-17fast-export: add a --show-original-ids option to show original namesLibravatar Elijah Newren1-0/+17
Knowing the original names (hashes) of commits can sometimes enable post-filtering that would otherwise be difficult or impossible. In particular, the desire to rewrite commit messages which refer to other prior commits (on top of whatever other filtering is being done) is very difficult without knowing the original names of each commit. In addition, knowing the original names (hashes) of blobs can allow filtering by blob-id without requiring re-hashing the content of the blob, and is thus useful as a small optimization. Once we add original ids for both commits and blobs, we may as well add them for tags too for completeness. Perhaps someone will have a use for them. This commit teaches a new --show-original-ids option to fast-export which will make it add a 'original-oid <hash>' line to blob, commits, and tags. It also teaches fast-import to parse (and ignore) such lines. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-17fast-export: add --reference-excluded-parents optionLibravatar Elijah Newren1-0/+11
git filter-branch has a nifty feature allowing you to rewrite, e.g. just the last 8 commits of a linear history git filter-branch $OPTIONS HEAD~8..HEAD If you try the same with git fast-export, you instead get a history of only 8 commits, with HEAD~7 being rewritten into a root commit. There are two alternatives: 1) Don't use the negative revision specification, and when you're filtering the output to make modifications to the last 8 commits, just be careful to not modify any earlier commits somehow. 2) First run 'git fast-export --export-marks=somefile HEAD~8', then run 'git fast-export --import-marks=somefile HEAD~8..HEAD'. Both are more error prone than I'd like (the first for obvious reasons; with the second option I have sometimes accidentally included too many revisions in the first command and then found that the corresponding extra revisions were not exported by the second command and thus were not modified as I expected). Also, both are poor from a performance perspective. Add a new --reference-excluded-parents option which will cause fast-export to refer to commits outside the specified rev-list-args range by their sha1sum. Such a stream will only be useful in a repository which already contains the necessary commits (much like the restriction imposed when using --no-data). Note from Peff: I think we might be able to do a little more optimization here. If we're exporting HEAD^..HEAD and there's an object in HEAD^ which is unchanged in HEAD, I think we'd still print it (because it would not be marked SHOWN), but we could omit it (by walking the tree of the boundary commits and marking them shown). I don't think it's a blocker for what you're doing here, but just a possible future optimization. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-17fast-export: ensure we export requested refsLibravatar Elijah Newren1-3/+13
If file paths are specified to fast-export and a ref points to a commit that does not touch any of the relevant paths, then that ref would sometimes fail to be exported. (This depends on whether any ancestors of the commit which do touch the relevant paths would be exported with that same ref name or a different ref name.) To avoid this problem, put *all* specified refs into extra_refs to start, and then as we export each commit, remove the refname used in the 'commit $REFNAME' directive from extra_refs. Then, in handle_tags_and_duplicates() we know which refs actually do need a manual reset directive in order to be included. This means that we do need some special handling for excluded refs; e.g. if someone runs git fast-export ^master master then they've asked for master to be exported, but they have also asked for the commit which master points to and all of its history to be excluded. That logically means ref deletion. Previously, such refs were just silently omitted from being exported despite having been explicitly requested for export. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-17fast-export: when using paths, avoid corrupt stream with non-existent markLibravatar Elijah Newren1-0/+20
If file paths are specified to fast-export and multiple refs point to a commit that does not touch any of the relevant file paths, then fast-export can hit problems. fast-export has a list of additional refs that it needs to explicitly set after exporting all blobs and commits, and when it tries to get_object_mark() on the relevant commit, it can get a mark of 0, i.e. "not found", because the commit in question did not touch the relevant paths and thus was not exported. Trying to import a stream with a mark corresponding to an unexported object will cause fast-import to crash. Avoid this problem by taking the commit the ref points to and finding an ancestor of it that was exported, and make the ref point to that commit instead. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-17fast-export: avoid dying when filtering by paths and old tags existLibravatar Elijah Newren1-0/+16
If --tag-of-filtered-object=rewrite is specified along with a set of paths to limit what is exported, then any tags pointing to old commits that do not contain any of those specified paths cause problems. Since the old tagged commit is not exported, fast-export attempts to rewrite such tags to an ancestor commit which was exported. If no such commit exists, then fast-export currently die()s. Five years after the tag rewriting logic was added to fast-export (see commit 2d8ad4691921, "fast-export: Add a --tag-of-filtered-object option for newly dangling tags", 2009-06-25), fast-import gained the ability to delete refs (see commit 4ee1b225b99f, "fast-import: add support to delete refs", 2014-04-20), so now we do have a valid option to rewrite the tag to. Delete these tags instead of dying. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-08Merge branch 'ma/fast-export-skip-merge-fix'Libravatar Junio C Hamano1-0/+18
"git fast-export" had a regression in v2.15.0 era where it skipped some merge commits in certain cases, which has been corrected. * ma/fast-export-skip-merge-fix: fast-export: fix regression skipping some merge-commits
2018-04-21fast-export: fix regression skipping some merge-commitsLibravatar Martin Ågren1-0/+18
7199203937 (object_array: add and use `object_array_pop()`, 2017-09-23) noted that the pattern `object = array.objects[--array.nr].item` could be abstracted as `object = object_array_pop(&array)`. Unfortunately, one of the conversions was horribly wrong. Between grabbing the last object (i.e., peeking at it) and decreasing the object count, the original code would sometimes return early. The updated code on the other hand, will always pop the last element, then maybe do the early return without doing anything with the object. The end result is that merge commits where all the parents have still not been exported will simply be dropped, meaning that they will be completely missing from the exported data. Re-add a commit when it is not yet time to handle it. An alternative that was considered was to peek-then-pop. That carries some risk with it since the peeking and popping need to act on the same object, in a concerted fashion. Add a test that would have caught this. Reported-by: Isaac Chou <Isaac.Chou@microfocus.com> Analyzed-by: Isaac Chou <Isaac.Chou@microfocus.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-28test: avoid pipes in git related commands for testLibravatar Pratik Karki1-26/+24
Avoid using pipes downstream of Git commands since the exit codes of commands upstream of pipes get swallowed, thus potentially hiding failure of those commands. Instead, capture Git command output to a file and apply the downstream command(s) to that file. Signed-off-by: Pratik Karki <predatoramigo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-29Merge branch 'jt/fast-export-copy-modify-fix'Libravatar Junio C Hamano1-1/+19
"git fast-export" with -M/-C option issued "copy" instruction on a path that is simultaneously modified, which was incorrect. * jt/fast-export-copy-modify-fix: fast-export: do not copy from modified file
2017-09-21fast-export: do not copy from modified fileLibravatar Jonathan Tan1-1/+19
When run with the "-C" option, fast-export writes 'C' commands in its output whenever the internal diff mechanism detects a file copy, indicating that fast-import should copy the given existing file to the given new filename. However, the diff mechanism works against the prior version of the file, whereas fast-import uses whatever is current. This causes issues when a commit both modifies a file and uses it as the source for a copy. Therefore, teach fast-export to refrain from writing 'C' when it has already written a modification command for a file. An existing test in t9350-fast-export is also fixed in this patch. The existing line "C file6 file7" copies the wrong version of file6, but it has coincidentally worked because file7 was subsequently overridden. Reported-by: Juraj Oršulić <juraj.orsulic@fer.hr> Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-10t3901: move supporting files into t/t3901/Libravatar Johannes Schindelin1-1/+1
The current convention is to either generate files on the fly in tests, or to use supporting files taken from a t/tNNNN/ directory (where NNNN matches the test's number, or the number of the test from which we borrow supporting files). The test t3901-i18n-patch.sh was obviously introduced before that convention was in full swing, hence its supporting files still lived in t/t3901-8859-1.txt and t/t3901-utf8.txt, respectively. Let's adjust to the current convention. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-12t9350-fast-export.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-3/+3
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do perl -i -pe 'BEGIN{undef $/;} s/`(.+?)`/\$(\1)/smg' "${_f}" done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21fast-export: add support to delete refsLibravatar Felipe Contreras1-0/+11
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-21fast-export: add new --refspec optionLibravatar Felipe Contreras1-0/+7
So that we can convert the exported ref names. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-29t: use perl instead of "$PERL_PATH" where applicableLibravatar Jeff King1-1/+1
As of the last commit, we can use "perl" instead of "$PERL_PATH" when running tests, as the former is now a function which uses the latter. As the shorter "perl" is easier on the eyes, let's switch to using it everywhere. This is not quite a mechanical s/$PERL_PATH/perl/ replacement, though. There are some places where we invoke perl from a script we generate on the fly, and those scripts do not have access to our internal shell functions. The result can be double-checked by running: ln -s /bin/false bin-wrappers/perl make test which continues to pass even after this patch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-07tests: use test_ln_s_add to remove SYMLINKS prerequisite (trivial cases)Libravatar Johannes Sixt1-3/+2
There are many instances where the treatment of symbolic links in the object model and the algorithms are tested, but where it is not necessary to actually have a symbolic link in the worktree. Make adjustments to the tests and remove the SYMLINKS prerequisite when appropriate in trivial cases, where "trivial" means: - merely a replacement of 'ln -s a b && git add b' by test_ln_s_add is needed; - a test for symbolic link on the file system can be split off (and remains protected by SYMLINKS); - existing code is equivalent to test_ln_s_add. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-15fast-export: add --signed-tags=warn-strip modeLibravatar John Keeping1-0/+6
This issues a warning while stripping signatures from signed tags, which allows us to use it as default behaviour for remote helpers which cannot specify how to handle signed tags. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-03fast-export: make sure updated refs get updatedLibravatar Felipe Contreras1-0/+15
When an object has already been exported (and thus is in the marks) it's flagged as SHOWN, so it will not be exported again, even if in a later time it's exported through a different ref. We don't need the object to be exported again, but we want the ref updated, which doesn't happen. Since we can't know if a ref was exported or not, let's just assume that if the commit was marked (flags & SHOWN), the user still wants the ref updated. IOW: If it's specified in the command line, it will get updated, regardless of whether or not the object was marked. So: % git branch test master % git fast-export $mark_flags master % git fast-export $mark_flags test Would export 'test' properly. Additionally, this fixes issues with remote helpers; now they can push refs whose objects have already been exported, and a few other issues as well. Update the tests accordingly. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-03fast-export: don't handle uninteresting refsLibravatar Felipe Contreras1-0/+30
They have been marked as UNINTERESTING for a reason, lets respect that. Currently the first ref is handled properly, but not the rest. Assuming that all the refs point at the same commit in the following example: % git fast-export master ^uninteresting ^foo ^bar reset refs/heads/bar from :0 reset refs/heads/foo from :0 reset refs/heads/uninteresting from :0 % git fast-export ^uninteresting ^foo ^bar master reset refs/heads/master from :0 reset refs/heads/bar from :0 reset refs/heads/foo from :0 Clearly this is wrong; the negative refs should be ignored. After this patch: % git fast-export ^uninteresting ^foo ^bar master # nothing % git fast-export master ^uninteresting ^foo ^bar # nothing And even more, it would only happen if the ref is pointing to exactly the same commit, but not otherwise: % git fast-export ^next next reset refs/heads/next from :0 % git fast-export ^next next^{commit} # nothing % git fast-export ^next next~0 # nothing % git fast-export ^next next~1 # nothing % git fast-export ^next next~2 # nothing The reason this happens is that before traversing the commits, fast-export checks if any of the refs point to the same object, and any duplicated ref gets added to a list in order to issue 'reset' commands after the traversing. Unfortunately, it's not even checking if the commit is flagged as UNINTERESTING. The fix of course, is to check it. However, in order to do it properly we need to get the UNINTERESTING flag from the command line, not from the commit object, because "^foo bar" will mark the commit 'bar' uninteresting if foo and bar points at the same commit. rev_cmdline_info, which was introduced exactly to handle this situation, contains all the information we need for get_tags_and_duplicates(), plus the ref flag. This way the rest of the positive refs will remain untouched; it's only the negative ones that change in behavior. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-29fast-export: fix comparison in testsLibravatar Felipe Contreras1-3/+3
First the expected, then the actual, otherwise the diff would be the opposite of what we want. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-26fast-export: avoid importing blob marksLibravatar Felipe Contreras1-0/+14
We want to be able to import, and then export, using the same marks, so that we don't push things that the other side already received. Unfortunately, fast-export doesn't store blobs in the marks, but fast-import does. This creates a mismatch when fast export is reusing a mark that was previously stored by fast-import. There is no point in one tool saving blobs, and the other not, but for now let's just check in fast-export that the objects are indeed commits. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-09Merge branch 'js/fast-export-paths-with-spaces'Libravatar Junio C Hamano1-1/+1
"git fast-export" produced an input stream for fast-import without properly quoting pathnames when they contain SPs in them. * js/fast-export-paths-with-spaces: fast-export: quote paths with spaces
2012-06-27fast-export: quote paths with spacesLibravatar Jay Soffian1-1/+1
A path containing a space must be quoted when used as an argument to either the copy or rename commands (because unlike other commands, the path is not the final thing on the line for those commands). Commit 6280dfdc3b (fast-export: quote paths in output, 2011-08-05) previously attempted to fix fast-export's quoting by passing all paths through quote_c_style(). However, that function does not consider the space to be a character which requires quoting, so let's special-case the space inside print_path(). This will cause space-containing paths to also be quoted in other commands where such quoting is not strictly necessary, but it does not hurt to do so. The test from 6280dfdc3b did not detect this because, while it does introduce renames in the export stream, it does not actually turn on rename detection, so they were presented as pairs of deletions/adds. Using "-M" reveals the bug. Signed-off-by: Jay Soffian <jaysoffian@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-24tests: enclose $PERL_PATH in double quotesLibravatar Junio C Hamano1-1/+1
Otherwise it will be split at a space after "Program" when it is set to "\\Program Files\perl" or something silly like that. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-12t: Replace 'perl' by $PERL_PATHLibravatar Vincent van Ravesteijn1-1/+1
GIT-BUILD-OPTIONS defines PERL_PATH to be used in the test suite. Only a few tests already actually use this variable when perl is needed. The other test just call 'perl' and it might happen that the wrong perl interpreter is used. This becomes problematic on Windows, when the perl interpreter that is compiled and installed on the Windows system is used, because this perl interpreter might introduce some unexpected LF->CRLF conversions. This patch makes sure that $PERL_PATH is used everywhere in the test suite and that the correct perl interpreter is used. Signed-off-by: Vincent van Ravesteijn <vfr@lyx.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-11tests: modernise style: more uses of test_line_countLibravatar Stefano Lattarini1-2/+2
Prefer: test_line_count <OP> COUNT FILE over: test $(wc -l <FILE) <OP> COUNT (or similar usages) in several tests. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-11Merge branch 'jk/fast-export-quote-path'Libravatar Junio C Hamano1-1/+27
* jk/fast-export-quote-path: fast-export: quote paths in output
2011-08-05fast-export: quote paths in outputLibravatar Jeff King1-1/+27
Many pathnames in a fast-import stream need to be quoted. In particular: 1. Pathnames at the end of an "M" or "D" line need quoting if they contain a LF or start with double-quote. 2. Pathnames on a "C" or "R" line need quoting as above, but also if they contain spaces. For (1), we weren't quoting at all. For (2), we put double-quotes around the paths to handle spaces, but ignored the possibility that they would need further quoting. This patch checks whether each pathname needs c-style quoting, and uses it. This is slightly overkill for (1), which doesn't actually need to quote many characters that vanilla c-style quoting does. However, it shouldn't hurt, as any implementation needs to be ready to handle quoted strings anyway. In addition to adding a test, we have to tweak a test which blindly assumed that case (2) would always use double-quotes, whether it needed to or not. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-09tests: add missing &&Libravatar Jonathan Nieder1-1/+1
Breaks in a test assertion's && chain can potentially hide failures from earlier commands in the chain. Commands intended to fail should be marked with !, test_must_fail, or test_might_fail. The examples in this patch do not require that. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31Merge branch 'en/d-f-conflict-fix'Libravatar Junio C Hamano1-0/+24
* en/d-f-conflict-fix: merge-recursive: Avoid excessive output for and reprocessing of renames merge-recursive: Fix multiple file rename across D/F conflict t6031: Add a testcase covering multiple renames across a D/F conflict merge-recursive: Fix typo Mark tests that use symlinks as needing SYMLINKS prerequisite t/t6035-merge-dir-to-symlink.sh: Remove TODO on passing test fast-import: Improve robustness when D->F changes provided in wrong order fast-export: Fix output order of D/F changes merge_recursive: Fix renames across paths below D/F conflicts merge-recursive: Fix D/F conflicts Add a rename + D/F conflict testcase Add additional testcases for D/F conflicts Conflicts: merge-recursive.c
2010-08-12Mark tests that use symlinks as needing SYMLINKS prerequisiteLibravatar Elijah Newren1-1/+1
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-19fast-export: Add a --full-tree optionLibravatar Elijah Newren1-0/+5
This option adds symmetry with fast-import, enabling it to also work with complete trees instead of just incremental changes. It works by issuing a 'deleteall' directive with each commit and then listing the full set of files that make up that commit, rather than just showing the list of files that have changed since the (first) parent commit. Note that this functionality is automatically turned on when using --import-marks together with path limiting in order to avoid dropping important but unchanged files. This functionality is desired when using hand-written filters along with 'fast-export | some-filter | fast-import' as it can be easier to write <some-filter> in terms of complete trees than incremental changes. We could avoid the need to add this option by simply always turning it on. While the end result would be identical, it would slow things down slightly by printing many more filenames per commit which goes somewhat against the 'fast' in 'fast-export'. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-19fast-export: Fix dropping of files with --import-marks and path limitingLibravatar Elijah Newren1-0/+9
Since fast-export operates by listing file changes since the (first) parent commit, when using --import-marks and path limiting and using a wider list of paths than in previous runs, files from the new path(s) will silently be omitted from the result unless or until a commit which explicitly changes those files. The resulting repository in such cases is broken and makes no sense. This commit fixes this by having fast-export work with complete trees instead of incremental changes (when both --import-marks and path limiting are used). It works by issuing a 'deleteall' directive with each commit and then listing the full set of files that make up that commit, rather than just showing the list of files that have changed since the (first) parent commit. Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-09fast-export: Fix output order of D/F changesLibravatar Elijah Newren1-1/+1
The fast-import stream format requires incremental changes which take place immediately, meaning that for D->F conversions all files below the relevant directory must be deleted before the resulting file of the same name is created. Reversing the order can result in fast-import silently deleting the file right after creating it, resulting in the file missing from the resulting repository. We correct this by first sorting the diff_queue_struct in depth-first order. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-09Add additional testcases for D/F conflictsLibravatar Elijah Newren1-0/+24
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-28t9350: fix careless use of "cd"Libravatar Junio C Hamano1-33/+38
Upon failure of any of these tests (or when a test that is marked as expecting a failure is fixed), we will end up running later tests in random places. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-07Rename t9301 to t9350, to make room for more fast-import testsLibravatar Johan Herland1-0/+374
Signed-off-by: Johan Herland <johan@herland.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>