summaryrefslogtreecommitdiff
path: root/perl/Git/SVN.pm
AgeCommit message (Collapse)AuthorFilesLines
2021-10-29tests: disable fsync everywhereLibravatar Eric Wong1-2/+15
The "GIT_TEST_FSYNC" environment variable now exists for disabling fsync() even on packfiles and other "critical" data. Running "make test -j8 NO_SVN_TESTS=1" on a noisy 8-core system on an HDD, test runtime drops from ~4 minutes down to ~3 minutes. Using "GIT_TEST_FSYNC=1" re-enables fsync() for comparison purposes. SVN interopability tests are minimally affected since SVN will still use fsync in various places. This will also be useful for 3rd-party tools which create throwaway git repositories of temporary data, but remains undocumented for end users. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-21git-svn: stop passing "-m" to "git rev-list"Libravatar Sergey Organov1-1/+1
rev-list doesn't utilize -m. It happens to eat it silently, so this bug went unnoticed. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-21perl: check for perl warnings while running testsLibravatar Jeff King1-1/+1
We set "use warnings" in most of our perl code to catch problems. But as the name implies, warnings just emit a message to stderr and don't otherwise affect the program. So our tests are quite likely to miss that warnings are being spewed, as most of them do not look at stderr. We could ask perl to make all warnings fatal, but this is likely annoying for non-developers, who would rather have a running program with a warning than something that refuses to work at all. So instead, let's teach the perl code to respect an environment variable (GIT_PERL_FATAL_WARNINGS) to increase the severity of the warnings. This can be set for day-to-day running if people want to be really pedantic, but the primary use is to trigger it within the test suite. We could also trigger that for every test run, but likewise even the tests failing may be annoying to distro builders, etc (just as -Werror would be for compiling C code). So we'll tie it to a special test-mode variable (GIT_TEST_PERL_FATAL_WARNINGS) that can be set in the environment or as a Makefile knob, and we'll automatically turn the knob when DEVELOPER=1 is set. That should give developers and CI the more careful view without disrupting normal users or packagers. Note that the mapping from the GIT_TEST_* form to the GIT_* form in test-lib.sh is necessary even if they had the same name: the perl scripts need it to be normalized to a perl truth value, and we also have to make sure it's exported (we might have gotten it from the environment, but we might also have gotten it from GIT-BUILD-OPTIONS directly). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-22perl: make SVN code hash independentLibravatar brian m. carlson1-35/+38
There are several places throughout git-svn that use various hard-coded constants. For matching object IDs, use the $oid variable. Compute the record size we use for our revision storage based on the object ID. When parsing the revision map format, use a wildcard in the pack format since we know that the data we're parsing is always exactly the record size. This lets us continue to use a constant for the pack format. Finally, update several comments to reflect the fact that an object ID may be of one of multiple sizes. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Acked-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-22perl: create and switch variables for hash constantsLibravatar brian m. carlson1-5/+5
git-svn has several variables for SHA-1 constants, including short hash values and full length hash values. Since these are no longer SHA-1 specific, let's start them with "oid" instead of "sha1". Add a constant, oid_length, which is the length of the hash algorithm in use in hex. We use the hex version because overwhelmingly that's what's used by git-svn. We don't currently set oid_length based on the repository algorithm, but we will in a future commit. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Acked-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-28git-svn: trim leading and trailing whitespaces in author nameLibravatar Tobias Klauser1-0/+4
In some cases, the svn author names might contain leading or trailing whitespaces, leading to messages such as: Author: user1 not defined in authors.txt (the trailing newline leads to the line break). The user "user1" is defined in authors.txt though, e.g. user1 = User <user1@example.com> Fix this by trimming the author name retreived from svn before using it in check_author. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Tobias Klauser <tklauser@distanz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-05git-svn: allow empty email-address using authors-prog and authors-fileLibravatar Andreas Heiduk1-7/+6
The email address in --authors-file and --authors-prog can be empty but git-svn translated it into a fictional email address in the form jondoe <jondoe@6aafaa21e0fb4338a68ab372a049893d> containing the SVN repository UUID. Now git-svn behaves like git-commit: If the email is *explicitly* set to the empty string using '<>', the commit does not contain an email address, only the name: jondoe <> Allowing to remove the email address *intentionally* prevents automatic systems from sending emails to those fictional addresses and avoids cluttering the log output with unnecessary stuff. Signed-off-by: Andreas Heiduk <asheiduk@gmail.com> Signed-off-by: Eric Wong <e@80x24.org>
2018-02-23perl: call timegm and timelocal with 4-digit yearLibravatar Bernhard M. Wiedemann1-1/+1
Amazingly, timegm(gmtime(0)) is only 0 before 2020 because perl's timegm deviates from GNU timegm(3) in how it handles years. man Time::Local says Whenever possible, use an absolute four digit year instead. with a detailed explanation about ambiguity of 2-digit years above that. Even though this ambiguity is error-prone with >50% of users getting it wrong, it has been like this for 20+ years, so we just use 4-digit years everywhere to be on the safe side. We add some extra logic to cvsimport because it allows 2-digit year input and interpreting an 18 as 1918 can be avoided easily and safely. Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-08git svn fetch: Create correct commit timestamp when using --localtimeLibravatar Urs Thuermann1-1/+1
In parse_svn_date() prepend the correct UTC offset to the timestamp returned. This is the offset in effect at the commit time instead of the offset in effect at calling time. Signed-off-by: Urs Thuermann <urs@isnogud.escape.de> Reviewed-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-23git-svn: escape backslashes in refnamesLibravatar Eric Wong1-1/+1
This brings git-svn refname escaping up-to-date with commit a4c2e69936df8dd0b071b85664c6cc6a4870dd84 ("Disallow '\' in ref names") from May 2009. Reported-by: Michael Fladischer <michael@fladi.at> Message-ID: <cb8cd9b1-9882-64d2-435d-40d0b2b82d59@fladi.at> Signed-off-by: Eric Wong <e@80x24.org>
2016-10-27Merge branch 'svn-cache' of git://bogomips.org/git-svnLibravatar Junio C Hamano1-1/+11
* 'svn-cache' of git://bogomips.org/git-svn: git-svn: do not reuse caches memoized for a different architecture
2016-10-27git-svn: do not reuse caches memoized for a different architectureLibravatar Gavin Lambert1-1/+11
Reusing cached data speeds up git-svn by quite a fair bit. However, if the YAML module is unavailable, the caches are written to disk in an architecture-dependent manner. That leads to problems when upgrading, say, from 32-bit to 64-bit Git for Windows. Let's just try to read those caches back if we detect the absence of the YAML module and the presence of the file, and delete the file if it could not be read back correctly. Note that the only way to catch the error when the memoized cache could not be read back is to put the call inside an `eval { ... }` block because it would die otherwise; the `eval` block should also return `1` in case of success explicitly since the function reading back the cached data does not return an appropriate value to test for success. This fixes https://github.com/git-for-windows/git/issues/233. [ew: import "retrieve" explictly, check unlink result] Signed-off-by: Gavin Lambert <github@mirality.co.nz> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Eric Wong <e@80x24.org>
2016-10-14git-svn: "git worktree" awarenessLibravatar Eric Wong1-9/+15
git-svn internals were previously not aware of repository layout differences for users of the "git worktree" command. Introduce this awareness by using "git rev-parse --git-path" instead of relying on outdated uses of GIT_DIR and friends. Thanks-to: Duy Nguyen <pclouds@gmail.com> Reported-by: Mathieu Arnold <mat@freebsd.org> Signed-off-by: Eric Wong <e@80x24.org>
2016-07-09git-svn: warn instead of dying when commit data is missingLibravatar Eric Wong1-2/+6
It is possible to have refs globbed by git-svn which stores data purely in git; gently skip those instead of dying and assuming user error. ref: http://mid.gmane.org/CALi1mtdtNF_GtzyPTbfb7N51wwxsFY7zm8hsgwxr3tHcZZboyg@mail.gmail.com Suggested-by: Jacob Godserv <jacobgodserv@gmail.com> Cc: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-22git-svn: skip mergeinfo handling with --no-follow-parentLibravatar Eric Wong1-9/+16
For repositories without parent following enabled, finding git parents through svn:mergeinfo or svk::parents can be expensive and pointless. Reported-by: Александр Овчинников <proff@proff.email> http://mid.gmane.org/4094761466408188@web24o.yandex.ru Signed-off-by: Eric Wong <e@80x24.org>
2016-05-08Git/SVN: die when there is no commit metadataLibravatar Christian Couder1-1/+2
When passing a bad --trunk option to `git svn clone`, like for example the same URL that we are cloning: C:\Windows\system32>git svn clone https://mycompany.svn.beanstalkapp.com/myproject --no-metadata -A c:\temp\svn_to_git_users.txt --trunk=https://mycompany.svn.beanstalkapp.com/myproject --tags=https://mycompany.svn.beanstalkapp.com/myproject/tags --branches=https://mycompany.svn.beanstalkapp.com/myproject/branches c:\code\Git_myproject One gets an "Use of uninitialized value $u in substitution (s///)" error: [...] W: +empty_dir: branches/20080918_DBDEPLOY/vendor/src/csharp/MS WCSF Contrib/src/Services W: +empty_dir: branches/20080918_DBDEPLOY/vendor/src/csharp/RealWorldControls/References r530 = c276e3b039d8e38759c6fb17443349732552d7a2 (refs/remotes/origin/trunk) Found possible branch point: https://mycompany.svn.beanstalkapp.com/myproject/trunk => https://mycompany.svn.beanstalkapp.com/myproject/branches/20080918_DBDEPLOY, 529 Use of uninitialized value $u in substitution (s///) at /mingw32/share/perl5/site_perl/Git/SVN.pm line 101. Use of uninitialized value $u in concatenation (.) or string at /mingw32/share/perl5/site_perl/Git/SVN.pm line 101. refs/remotes/origin/trunk: 'https://mycompany.svn.beanstalkapp.com/myproject' not found in '' C:\Windows\system32> Let's fix that by just die()ing when we have an uninitialized value because we cannot get commit metadata from a ref. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Eric Wong <e@80x24.org>
2015-11-10git-svn: improve rebase/mkdirs performanceLibravatar Dair Grant1-8/+76
Processing empty_dir directives becomes extremely slow for svn repositories with a large enough history. This is due to using a single hash to store the list of empty directories, with the expensive step being purging items from that hash using grep+delete. Storing directories in a hash of hashes improves the performance of this purge step and removes a potentially lengthy delay after every rebase/mkdirs command. The svn repository with this behaviour has 110K commits with unhandled.log containing 170K empty_dir directives. This takes 10 minutes to process when using a single hash, vs 3 seconds with a hash of hashes. Signed-off-by: Dair Grant <dair@feralinteractive.com> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2015-02-26Merge branch 'svn-maint-fixes' into svn-fixesLibravatar Junio C Hamano1-1/+2
* svn-maint-fixes: Git::SVN::*: avoid premature FileHandle closure git-svn: fix localtime=true on non-glibc environments
2015-02-26git-svn: fix localtime=true on non-glibc environmentsLibravatar Ryuichi Kokubo1-1/+2
git svn uses POSIX::strftime('%s', $sec, $min, ...) to make unix epoch time. But lowercase %s formatting character is a GNU extention. This causes problem in git svn fetch --localtime on non-glibc systems, such as msys or cygwin. Using Time::Local::timelocal($sec, $min, ...) fixes it. Signed-off-by: Ryuichi Kokubo <ryu1kkb@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net> Notes: lowercase %s format character in strftime is a GNU extension and not widely supported. POSIX::strftime affected by underlying crt's strftime because POSIX::strftime just calls crt's one. Time::Local is good function to replace POSIX::strftime because it's a perl core module function. Document about Time::Local. http://perldoc.perl.org/Time/Local.html These are specifications of strftime. The GNU C Library Reference Manual. http://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html perl POSIX module's strftime document. It does not have '%s'. http://perldoc.perl.org/POSIX.html strftime document of Microsort Windows C Run-Time library. https://msdn.microsoft.com/en-us/library/fe06s4ak.aspx The Open Group's old specification does not have '%s' too. http://pubs.opengroup.org/onlinepubs/007908799/xsh/strftime.html On my environment, following problems happened. - msys : git svn fetch does not progress at all with perl.exe consuming CPU. - cygwin : git svn fetch progresses but time stamp information is dropped. Every commits have unix epoch timestamp. I would like to thank git developer and contibutors. git helps me so much everyday. Thank you. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-26git-svn: lazy load some modulesLibravatar Eric Wong1-9/+13
We can delay loading some modules until we need them for uncommon code paths. For example, persistent memoization is not often needed, so we can avoid loading the modules for it until we encounter svn::mergeinfo during fetch. This gives a tiny reduction in syscalls (from 15641 to 15305) when running "git svn info" and counting via "strace -fc". Further, more invasive work will be needed to noticeably improve performance. Signed-off-by: Eric Wong <normalperson@yhbt.net>
2015-01-15Git::SVN: handle missing ref_id case correctlyLibravatar Ramkumar Ramachandra1-1/+1
ref_id should not match "refs/remotes/". [ew: dropped initial hunk for GIT_SVN_ID at Ramkumar's request] Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-30git-svn: add space after "W:" prefix in warningLibravatar Eric Wong1-4/+2
And minor reformatting while we're in the area. Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-30git-svn: (cleanup) remove editor param passingLibravatar Eric Wong1-6/+4
Neither find_extra_svk_parents or find_extra_svn_parents ever used the `$ed' parameter. Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-27git-svn: disable _rev_list memoizationLibravatar Eric Wong1-10/+0
This memoization appears unneeded as the check_cherry_pick2 cache is in front of it does enough. With this change applied, importing from local svn+ssh and http copies of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2]. This 60M measurement is only for the git-svn Perl process itself and does not include memory used by git subprocesses accessing large packs (subprocess memory usage _is_ measured by my time(1) tool). Before this change, an import took longer (2:20) on svn+ssh:// but git-svn used around 240MB during the imports. Worse yet, git-svn ballooned to over 400M when writing out the cache to the filesystem. I also tried removing memoization for `has_no_changes', too, but a local copy of the R repository(*) was not close to finishing within 10 hours on my system. [1] http://svn.r-project.org/R [2] file:// repos causes libsvn to use more memory internally Signed-off-by: Eric Wong <normalperson@yhbt.net> Cc: Hin-Tak Leung <htl10@users.sourceforge.net>
2014-10-24git-svn: remove mergeinfo rev cachingLibravatar Eric Wong1-21/+9
This should further reduce memory usage from the new mergeinfo speedups without hurting performance too much, assuming reasonable latency to the SVN server. Cc: Hin-Tak Leung <htl10@users.sourceforge.net> Suggested-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-24git-svn: cache only mergeinfo revisionsLibravatar Eric Wong1-14/+8
This should reduce excessive memory usage from the new mergeinfo caches without hurting performance too much, assuming reasonable latency to the SVN server. Cc: Hin-Tak Leung <htl10@users.sourceforge.net> Suggested-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-24git-svn: reduce check_cherry_pick cache overheadLibravatar Eric Wong1-13/+15
We do not need to store entire lists of commits, only the number of incomplete and the first commit for reference. This reduces the amount of data we need to store in memory and on disk stores. Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-24git-svn: only look at the root path for svn:mergeinfoLibravatar Jakob Stoklund Olesen1-16/+13
Subversion can put mergeinfo on any sub-directory to track cherry-picks. Since cherry-picks are not represented explicitly in git, git-svn should just ignore it. Signed-off-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-10-24git-svn: only look at the new parts of svn:mergeinfoLibravatar Jakob Stoklund Olesen1-12/+72
In a Subversion repository where many feature branches are merged into a trunk, the svn:mergeinfo property can grow very large. This severely slows down git-svn's make_log_entry() because it is checking all mergeinfo entries every time the property changes. In most cases, the additions to svn:mergeinfo since the last commit are pretty small, and there is nothing to gain by checking merges that were already checked for the last commit in the branch. Add a mergeinfo_changes() function which computes the set of interesting changes to svn:mergeinfo since the last commit. Filter out merged branches whose ranges haven't changed, and remove a common prefix of ranges from other merged branches. This speeds up "git svn fetch" by several orders of magnitude on a large repository where thousands of feature branches have been merged. Signed-off-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2014-04-17SVN.pm::parse_svn_date: allow timestamps with a single-digit hourLibravatar RomanBelinsky1-1/+1
Some broken subversion server gives timestamps with only one digit in the hour part, like this: 2014-01-07T5:01:02.048176Z Loosen the regexp that expected to see two-digit hour, minute and second parts to accept a single-digit hour (but not minute or second). Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-31code and test: fix misuses of "nor"Libravatar Justin Lebar1-2/+2
Signed-off-by: Justin Lebar <jlebar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-31comments: fix misuses of "nor"Libravatar Justin Lebar1-1/+1
Signed-off-by: Justin Lebar <jlebar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-23git-svn: memoize _rev_list and rebuildLibravatar lin zuojian1-3/+38
According to profile data, _rev_list and rebuild consume a large portion of time. Memoize the results of _rev_list and memoize rebuild internals to avoid subprocess invocation. When importing 15152 revisions on a LAN, time improved from 10 hours to 3-4 hours. Signed-off-by: lin zuojian <manjian2006@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2013-03-08git svn: consistent spacing after "W:" in warningsLibravatar Eric Wong1-1/+1
All other instances of "W:"-prefixed warning messages have a space after the "W:" to help with readability. Signed-off-by: Eric Wong <normalperson@yhbt.net>
2013-03-08git svn: ignore partial svn:mergeinfoLibravatar Jan Pešta1-0/+5
Currently this is cosmetic change - the merges are ignored, becuase the methods (lookup_svn_merge, find_rev_before, find_rev_after) are failing on comparing text with number. See http://www.open.collab.net/community/subversion/articles/merge-info.html Extract: The range r30430:30435 that was added to 1.5.x in this merge has a '*' suffix for 1.5.x\www. This '*' is the marker for a non-inheritable mergeinfo range. The '*' means that only the path on which the mergeinfo is explicitly set has had this range merged into it. Signed-off-by: Jan Pesta <jan.pesta@certicon.cz> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2013-02-14Merge branch 'bw/get-tz-offset-perl'Libravatar Junio C Hamano1-10/+2
* bw/get-tz-offset-perl: cvsimport: format commit timestamp ourselves without using strftime perl/Git.pm: fix get_tz_offset to properly handle DST boundary cases Move Git::SVN::get_tz to Git::get_tz_offset
2013-02-09Move Git::SVN::get_tz to Git::get_tz_offsetLibravatar Ben Walton1-10/+2
This function has utility outside of the SVN module for any routine that needs the equivalent of GNU strftime's %z formatting option. Move it to the top-level Git.pm so that non-SVN modules don't need to import the SVN module to use it. The rename makes the purpose of the function clearer. Signed-off-by: Ben Walton <bdwalton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-24git-svn: cleanup sprintf usage for uppercasing hexLibravatar Eric Wong1-2/+2
We do not need to call uc() separately for sprintf("%x") as sprintf("%X") is available. Signed-off-by: Eric Wong <normalperson@yhbt.net> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
2012-10-05Git::SVN: rename private path fieldLibravatar Jonathan Nieder1-2/+2
All users of $gs->{path} should have been converted to use the accessor by now. Check our work by renaming the underlying variable to break callers that try to use it directly. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-10-05git-svn.perl: keep processing all commits in parents_excludeLibravatar Steven Walter1-1/+0
This fixes a bug where git finds the incorrect merge parent. Consider a repository with trunk, branch1 of trunk, and branch2 of branch1. Without this change, git interprets a merge of branch2 into trunk as a merge of branch1 into trunk. Signed-off-by: Steven Walter <stevenrwalter@gmail.com> Reviewed-by: Sam Vilain <sam@vilain.net> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-10-05git-svn.perl: consider all ranges for a given merge, instead of only tip-by-tipLibravatar Steven Walter1-5/+3
Consider the case where you have trunk, branch1 of trunk, and branch2 of branch1. trunk is merged back into branch2, and then branch2 is reintegrated into trunk. The merge of branch2 into trunk will have svn:mergeinfo property references to both branch1 and branch2. When git-svn fetches the commit that merges branch2 (check_cherry_pick), it is necessary to eliminate the merged contents of branch1 as well as branch2, or else the merge will be incorrectly ignored as a cherry-pick. Signed-off-by: Steven Walter <stevenrwalter@gmail.com> Reviewed-by: Sam Vilain <sam@vilain.net> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-22Merge branch 'ms/git-svn-1.7'Libravatar Junio C Hamano1-63/+111
A series by Michael Schwern via Eric to update git-svn to revamp the way URLs are internally passed around, to make it work with SVN 1.7. * ms/git-svn-1.7: git-svn: remove ad-hoc canonicalizations git-svn: canonicalize newly-minted URLs git-svn: introduce add_path_to_url function git-svn: canonicalize earlier git-svn: replace URL escapes with canonicalization git-svn: attempt to mimic SVN 1.7 URL canonicalization t9107: fix typo t9118: workaround inconsistency between SVN versions Git::SVN{,::Ra}: canonicalize earlier git-svn: path canonicalization uses SVN API Git::SVN::Utils: remove irrelevant comment git-svn: add join_paths() to safely concatenate paths git-svn: factor out _collapse_dotdot function git-svn: use SVN 1.7 to canonicalize when possible git-svn: move canonicalization to Git::SVN::Utils use Git::SVN{,::RA}->url accessor globally use Git::SVN->path accessor globally Git::SVN::Ra: use accessor for URLs Git::SVN: use accessor for URLs internally Git::SVN: use accessors internally for path
2012-08-10git svn: reset invalidates the memoized mergeinfo cachesLibravatar Peter Baumann1-2/+25
Since v1.7.0-rc2~11 (git-svn: persistent memoization, 2010-01-30), git-svn has maintained some private per-repository caches in .git/svn/.caches to avoid refetching and recalculating some mergeinfo-related information with every 'git svn fetch'. This memoization can cause problems, e.g consider the following case: SVN repo: ... - a - b - c - m <- trunk \ / d - e <- branch1 The Git import of the above repo is at commit 'a' and doesn't know about the branch1. In case of an 'git svn rebase', only the trunk of the SVN repo is imported. During the creation of the git commit 'm', git svn uses the svn:mergeinfo property and tries to find the corresponding git commit 'e' to create 'm' with 'c' and 'e' as parents. But git svn rebase only imports the current branch so commit 'e' is not imported. Therefore git svn fails to create commit 'm' as a merge commit, because one of its parents is not known to git. The imported history looks like this: ... - a - b - c - m <- trunk A later 'git svn fetch' to import all branches can't rewrite the commit 'm' to add 'e' as a parent and to make it a real git merge commit, because it was already imported. That's why the imported history misses the merge and looks like this: ... - a - b - c - m <- trunk \ d - e <- branch1 Right now the only known workaround for importing 'm' as a merge is to force reimporting 'm' again from SVN, e.g. via $ git svn reset --revision $(git find-rev $c) $ git svn fetch Sadly, this is where the behavior has regressed: git svn reset doesn't invalidate the old mergeinfo cache, which is no longer valid for the reimport, which leads to 'm' beeing imprted with only 'c' as parent. As solution to this problem, this commit invalidates the mergeinfo cache to force correct recalculation of the parents. During development of this patch, several ways for invalidating the cache where considered. One of them is to use Memoize::flush_cache, which will call the CLEAR method on the underlying Memoize persistency implementation. Sadly, neither Memoize::Storable nor the newer Memoize::YAML module introduced in 68f532f4ba888 could optionally be used implement the CLEAR method, so this is not an option. Reseting the internal hash used to store the memoized values has the same problem, because it calls the non-existing CLEAR method of the underlying persistency layer, too. Considering this and taking into account the different implementations of the memoization modules, where Memoize::Storable is not in our control, implementing the missing CLEAR method is not an option, at least not if Memoize::Storable is still used. Therefore the easiest solution to clear the cache is to delete the files on disk in 'git svn reset'. Normally, deleting the files behind the back of the memoization module would be problematic, because the in-memory representation would still exist and contain wrong data. Fortunately, the memoization is active in memory only for a small portion of the code. Invalidating the cache by deleting the files on disk if it isn't active should be safe. Signed-off-by: Peter Baumann <waste.manager@gmx.de> Signed-off-by: Steven Walter <stevenrwalter@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02git-svn: remove ad-hoc canonicalizationsLibravatar Michael G. Schwern1-1/+0
[ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02git-svn: canonicalize newly-minted URLsLibravatar Michael G. Schwern1-5/+14
Go through all the spots that use the new add_path_to_url() to make a new URL and canonicalize them. * copyfrom_path has to be canonicalized else find_parent_branch will get confused * due to the `canonicalize_url($full_url) ne $full_url)` line of logic in gs_do_switch(), $full_url is left alone until after. At this point SVN 1.7 passes except for 3 tests in t9100-git-svn-basic.sh that look like an SVN bug to do with symlinks. [ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02git-svn: introduce add_path_to_url functionLibravatar Michael G. Schwern1-18/+15
Remove the ad-hoc versions. This is mostly to normalize the process and ensure the URLs produced don't have double slashes or anything. Also provides a place to fix the corner case where a file path contains a percent sign. [ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02git-svn: replace URL escapes with canonicalizationLibravatar Michael G. Schwern1-1/+1
The old hand-rolled URL escape functions were inferior to canonicalization functions. Continuing to move towards getting everything canonicalizing the same way. * Git::SVN->init_remote_config and Git::SVN::Ra->minimize_url both have to canonicalize the same way else init_remote_config will incorrectly think they're different URLs causing t9107-git-svn-migrate.sh to fail. [ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02git-svn: attempt to mimic SVN 1.7 URL canonicalizationLibravatar Michael G. Schwern1-2/+2
Previously, our URL canonicalization didn't do much of anything. Now it actually escapes and collapses slashes. This is mostly a cut & paste of escape_url from git-svn. This is closer to how SVN 1.7's canonicalization behaves. Doing it with 1.6 lets us chase down some problems caused by more effective canonicalization without having to deal with all the other 1.7 issues on top of that. * Remote URLs have to be canonicalized otherwise Git::SVN->find_existing_remote will think they're different. * The SVN remote is now written to the git config canonicalized. That should be ok. Adjust a test to account for that. [ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02Git::SVN{,::Ra}: canonicalize earlierLibravatar Michael G. Schwern1-2/+4
This canonicalizes paths and urls as early as possible so we don't have to remember to do it at the point of use. It will fix a swath of SVN 1.7 problems in one go. Its ok to double canonicalize things. SVN 1.7 still fails, still not worrying about that. [ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-08-02git-svn: add join_paths() to safely concatenate pathsLibravatar Michael G. Schwern1-4/+6
Otherwise you might wind up with things like... my $path1 = undef; my $path2 = 'foo'; my $path = $path1 . '/' . $path2; creating '/foo'. Or this... my $path1 = 'foo/'; my $path2 = 'bar'; my $path = $path1 . '/' . $path2; creating 'foo//bar'. Could have used File::Spec, but I'm shying away from it due to SVN 1.7's pickiness about paths. Felt it would be better to have our own we can control completely. [ew: commit title] Signed-off-by: Eric Wong <normalperson@yhbt.net>