summaryrefslogtreecommitdiff
path: root/git-p4.py
AgeCommit message (Collapse)AuthorFilesLines
2020-03-25Merge branch 'yz/p4-py3'Libravatar Junio C Hamano1-95/+140
Update "git p4" to work with Python 3. * yz/p4-py3: ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere git-p4: use python3's input() everywhere git-p4: simplify regex pattern generation for parsing diff-tree git-p4: use dict.items() iteration for python3 compatibility git-p4: use functools.reduce instead of reduce git-p4: fix freezing while waiting for fast-import progress git-p4: use marshal format version 2 when sending to p4 git-p4: open .gitp4-usercache.txt in text mode git-p4: convert path to unicode before processing them git-p4: encode/decode communication with git for python3 git-p4: encode/decode communication with p4 for python3 git-p4: remove string type aliasing git-p4: change the expansion test from basestring to list git-p4: make python2.7 the oldest supported version
2020-01-30git-p4: avoid leak of file handle when cloningLibravatar Luke Diamand1-1/+2
Spotted by Eric Sunshine: https://public-inbox.org/git/CAPig+cRx3hG64nuDie69o_gdX39F=sR6D8LyA7J1rCErgu0aMA@mail.gmail.com/ Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: check for access to remote host earlierLibravatar Luke Diamand1-0/+2
Check we can talk to the remote host before starting the git-fastimport subchild. Otherwise we fail to connect, and then exit, leaving git-fastimport still running since we did not wait() for it. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: cleanup better on error exitLibravatar Luke Diamand1-15/+28
After an error, git-p4 calls die(). This just exits, and leaves child processes still running. Instead of calling die(), raise an exception and catch it where the child process(es) (git-fastimport) are created. This was analyzed in detail here: https://public-inbox.org/git/20190227094926.GE19739@szeder.dev/ This change does not address the particular issue of p4CmdList() invoking a subchild and not waiting for it on error. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: create helper function importRevisions()Libravatar Luke Diamand1-64/+68
This makes it easier to try/catch around this block of code to ensure cleanup following p4 failures is handled properly. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: disable some pylint warnings, to get pylint output to something ↵Libravatar Luke Diamand1-0/+8
manageable pylint is incredibly useful for finding bugs, but git-p4 has never used it, so there are a lot of warnings that while important, don't actually result in bugs. Let's turn those off for now, so we can get some useful output. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: add P4CommandException to report errors talking to PerforceLibravatar Luke Diamand1-0/+8
Currently when there is a P4 error, git-p4 calls die() which just exits. This then leaves the git-fast-import process still running, and can even leave p4 itself still running. As a result, git-p4 fails to exit cleanly. This is a particular problem for people running the unit tests in regression. Use this exception to report errors upwards, cleaning up as the error propagates. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: make closeStreams() idempotentLibravatar Luke Diamand1-0/+3
Ensure that we can safely call self.closeStreams() multiple times, and can also call it even if there is no git fast-import stream at all. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use python3's input() everywhereLibravatar Yang Zhao1-2/+12
Python3 deprecates raw_input() from 2.7 and replaced it with input(). Since we do not need 2.7's input() semantics, `raw_input()` is aliased to `input()` for easy forward compatability. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: simplify regex pattern generation for parsing diff-treeLibravatar Yang Zhao1-7/+6
It is not clear why a generator was used to create the regex used to parse git-diff-tree output; I assume an early implementation required it, but is not part of the mainline change. Simply use a lazily initialized global instead. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use dict.items() iteration for python3 compatibilityLibravatar Yang Zhao1-1/+1
Python3 uses dict.items() instead of .iteritems() to provide iteratoration over dict. Although items() is technically less efficient for python2.7 (allocates a new list instead of simply iterating), the amount of data involved is very small and the penalty negligible. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use functools.reduce instead of reduceLibravatar Yang Zhao1-1/+2
For python3, reduce() has been moved to functools.reduce(). This is also available in python2.7. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: fix freezing while waiting for fast-import progressLibravatar Yang Zhao1-0/+1
As part of its importing process, git-p4 sends a `checkpoint` followed immediately by `progress` to fast-import to force synchronization. Due to buffering, it is possible for the `progress` command to not be flushed before git-p4 begins to wait for the corresponding response. This causes the script to freeze completely, and is consistently observable at least on python-3.6.9. Make sure this command sequence is completely flushed before waiting. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use marshal format version 2 when sending to p4Libravatar Yang Zhao1-1/+2
p4 does not appear to understand marshal format version 3 and above. Version 2 was the latest supported by python-2.7. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: open .gitp4-usercache.txt in text modeLibravatar Yang Zhao1-2/+2
Opening .gitp4-usercache.txt in text mode makes python 3 happy without explicitly adding encoding and decoding. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: convert path to unicode before processing themLibravatar Yang Zhao1-25/+44
P4 allows essentially arbitrary encoding for path data while we would perfer to be dealing only with unicode strings. Since path data need to survive round-trip back to p4, this patch implements the general policy that we store path data as-is, but decode them to unicode before doing any non-trivial processing. A new `decode_path()` method is provided that generally does the correct conversion, taking into account `git-p4.pathEncoding` configuration. For python2.7, path strings will be left as-is if it only contains ASCII characters. For python3, decoding is always done so that we have str objects. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: encode/decode communication with git for python3Libravatar Yang Zhao1-4/+17
Under python3, calls to write() on the stream to `git fast-import` must be encoded. This patch wraps the IO object such that this encoding is done transparently. Conversely, any text data read from subprocesses must also be decoded before running through the rest of the pipeline. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: encode/decode communication with p4 for python3Libravatar Yang Zhao1-13/+46
The marshalled dict in the response given on STDOUT by p4 uses `str` for keys and string values. When run using python3, these values are deserialized as `bytes`, leading to a whole host of problems as the rest of the code assumes `str` is used throughout. This patch changes the deserialization behaviour such that, as much as possible, text output from p4 is decoded to native unicode strings. Exceptions are made for the field `data` as it is usually arbitrary binary data. `depotFile[0-9]*`, `path`, and `clientFile` are also exempt as they contain path strings not encoded with UTF-8, and must survive round-trip back to p4. Conversely, text data being piped to p4 must always be encoded when running under python3. encode_text_stream() and decode_text_stream() were added to make these transformations more convenient. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: remove string type aliasingLibravatar Yang Zhao1-16/+0
Now that python2.7 is the minimum required version and we no longer use the basestring type, it is not necessary to use type aliasing to ensure python3 compatibility. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: change the expansion test from basestring to listLibravatar Ben Keene1-9/+9
Python 3 handles strings differently than Python 2.7. Since Python 2 is reaching it's end of life, a series of changes are being submitted to enable python 3.5 and following support. The current code fails basic tests under python 3.5. Some codepaths can represent a command line the program internally prepares to execute either as a single string (i.e. each token properly quoted, concatenated with $IFS) or as a list of argv[] elements, and there are 9 places where we say "if X is isinstance(_, basestring), then do this thing to handle X as a command line in a single string; if not, X is a command line in a list form". This does not work well with Python 3, as there is no basestring (everything is Unicode now), and even with Python 2, it was not an ideal way to tell the two cases apart, because an internally formed command line could have been in a single Unicode string. Flip the check to say "if X is not a list, then handle X as a command line in a single string; otherwise treat it as a command line in a list form". This will get rid of references to 'basestring', to migrate the code ready for Python 3. Thanks-to: Junio C Hamano <gitster@pobox.com> Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: make python2.7 the oldest supported versionLibravatar Yang Zhao1-18/+2
Python2.6 and earlier have been end-of-life'd for many years now, and we actually already use 2.7-only features in the code. Make the version check reflect current realities. This also removes the need to explicitly define CalledProcessError if it's not available. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-02Merge branch 'bk/p4-misc-usability'Libravatar Junio C Hamano1-32/+42
Miscellaneous small UX improvements on "git-p4". * bk/p4-misc-usability: git-p4: show detailed help when parsing options fail git-p4: yes/no prompts should sanitize user text
2019-12-16git-p4: show detailed help when parsing options failLibravatar Ben Keene1-1/+6
When a user provides invalid parameters to git-p4, the program reports the failure but does not provide the correct command syntax. Add an exception handler to the command-line argument parser to display the command's specific command line parameter syntax when an exception is thrown. Rethrow the exception so the current behavior is retained. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-16git-p4: yes/no prompts should sanitize user textLibravatar Ben Keene1-31/+36
When prompting the user interactively for direction, the tests are not forgiving of user input format. For example, the first query asks for a yes/no response. If the user enters the full word "yes" or "no" or enters a capital "Y" the test will fail. Create a new function, prompt(prompt_text) where * prompt_text is the text prompt for the user * returns a single character where valid return values are found by inspecting prompt_text for single characters surrounded by square brackets This new function must prompt the user for input and sanitize it by converting the response to a lower case string, trimming leading and trailing spaces, and checking if the first character is in the list of choices. If it is, return the first letter. Change the current references to raw_input() to use this new function. Since the method requires the returned text to be one of the available choices, remove the loop from the calling code that handles response verification. Thanks-to: Denton Liu <liu.denton@gmail.com> Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-11git-p4: honor lfs.storage configuration variableLibravatar r.burenkov1-2/+8
"git lfs" allows users to specify the custom storage location with the configuration variable `lfs.storage`, but when interacting with GitLFS pointers, "git p4" always uses the hardcoded default that is the `.git/lfs/` directory, without paying attention to the configuration. Use the value configured in `lfs.storage`, if exists, as all the "git" operations do, for consistency. Signed-off-by: r.burenkov <panzercheg@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-06git-p4: auto-delete named temporary fileLibravatar Philip.McGraw1-7/+6
Avoid double-open exceptions on Windows platform when calculating for lfs compressed size threshold (git-p4.largeFileCompressedThreshold) comparisons. Take new approach using the NamedTemporaryFile() file-like object as input to the ZipFile() which auto-deletes after implicit close leaving with scope. Original code had double-open exception on Windows platform because file still open from NamedTemporaryFile() using generated filename instead of object. Thanks to Andrey for patiently suggesting several iterations on this change for avoiding exceptions! Also print error details after resulting IOError to make debugging cause of exception less mysterious when it has nothing to do with "git version recent enough." Signed-off-by: Philip.McGraw <Philip.McGraw@bentley.com> Reviewed-by: Andrey Mazo <ahippo@yandex.com> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-09Merge branch 'am/p4-branches-excludes'Libravatar Junio C Hamano1-17/+27
"git p4" update. * am/p4-branches-excludes: git-p4: respect excluded paths when detecting branches git-p4: add failing test for "git-p4: respect excluded paths when detecting branches" git-p4: don't exclude other files with same prefix git-p4: add failing test for "don't exclude other files with same prefix" git-p4: don't groom exclude path list on every commit git-p4: match branches case insensitively if configured git-p4: add failing test for "git-p4: match branches case insensitively if configured" git-p4: detect/prevent infinite loop in gitCommitByP4Change()
2019-06-17Merge branch 'mm/p4-unshelve-windows-fix'Libravatar Junio C Hamano1-1/+1
The command line to invoke a "git cat-file" command from inside "git p4" was not properly quoted to protect a caret and running a broken command on Windows, which has been corrected. * mm/p4-unshelve-windows-fix: p4 unshelve: fix "Not a valid object name HEAD0" on Windows
2019-05-28p4 unshelve: fix "Not a valid object name HEAD0" on WindowsLibravatar Mike Mueller1-1/+1
git p4 unshelve was failing with these errors: fatal: Not a valid object name HEAD0 Command failed: git cat-file commit HEAD^0 (git version 2.21.0.windows.1, python 2.7.16) The pOpen call used by git-p4 to invoke the git command can take either a string or an array as a first argument. The array form is preferred because platform-specific escaping of special characters will be handled automatically.(https://docs.python.org/2/library/subprocess.html) The extractLogMessageFromGitCommit method was, however, using the string form and so the caret (^) character in the HEAD^0 argument was not being escaped on Windows. The caret happens to be the escape character, which is why the git command was receiving HEAD0. The behaviour can be confirmed by typing ECHO HEAD^0 at the command- prompt, which emits HEAD0. The solution is simply to use the array format of passing the command to fOpen, which is recommended and used in other parts of this code anyway. Signed-off-by: Mike Mueller <mike.mueller@moodys.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-28git-p4: allow unshelving of branched filesLibravatar Simon Williams1-1/+1
When unshelving a changelist, git-p4 tries to work out the appropriate parent commit in a given branch (default: HEAD). To do this, it looks at the state of any pre-existing files in the target Perforce branch, omitting files added in the shelved changelist. Currently, only files added (or move targets) are classed as new. However, files integrated from other branches (i.e. a 'branch' action) also need to be considered as added, for this purpose. Signed-off-by: Simon Williams <simon@no-dns-yet.org.uk> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: respect excluded paths when detecting branchesLibravatar Mazo, Andrey1-2/+1
Currently, excluded paths are only handled in the following cases: * no branch detection; * branch detection with using clientspec. However, excluded paths are not respected in case of branch detection without using clientspec. Fix this by consulting the list of excluded paths when splitting files across branches. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: don't exclude other files with same prefixLibravatar Mazo, Andrey1-7/+14
Make sure not to exclude files unintentionally if exclude paths are specified without a trailing /. I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file". Do this by ensuring that paths without a trailing "/" are only matched completely. Also, abort path search on the first match as a micro-optimization. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: don't groom exclude path list on every commitLibravatar Mazo, Andrey1-5/+7
Currently, `cloneExclude` array is being groomed (by removing trailing "...") on every changeset. (since `extractFilesFromCommit()` is called on every imported changeset) As a micro-optimization, do it once while parsing arguments. Also, prepend "/" and remove trailing "..." at the same time. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: match branches case insensitively if configuredLibravatar Mazo, Andrey1-2/+2
git-p4 knows how to handle case insensitivity in file paths if core.ignorecase is set. However, when determining a branch for a file, it still does a case-sensitive prefix match. This may result in some file changes to be lost on import. For example, given the following commits 1. add //depot/main/file1 2. add //depot/DirA/file2 3. add //depot/dira/file3 4. add //depot/DirA/file4 and "branchList = main:DirA" branch mapping, commit 3 will be lost. So, do branch search case insensitively if running with core.ignorecase set. Teach splitFilesIntoBranches() to use the p4PathStartsWith() function for path prefix matches instead of always case-sensitive match. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: detect/prevent infinite loop in gitCommitByP4Change()Libravatar Mazo, Andrey1-1/+3
Under certain circumstances, gitCommitByP4Change() can enter an infinite loop resulting in `git p4 sync` hanging forever. The problem is that `git rev-list --bisect <latest> ^<earliest>` can return `<latest>`, which would result in reinspecting <latest> and potentially an infinite loop. This can happen when importing just a subset of P4 repository and/or with explicit "--changesfile" option. A real-life example: """ looking in ref refs/remotes/p4/mybranch for change 26894 using bisect... Reading pipe: git rev-parse refs/remotes/p4/mybranch trying: earliest latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git cat-file commit 147f5d3292af2e1cc4a56a7b96db845144c68486 current change 25339 trying: earliest ^147f5d3292af2e1cc4a56a7b96db845144c68486 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^147f5d3292af2e1cc4a56a7b96db845144c68486 Reading pipe: git cat-file commit 51db83df9d588010d0bd995641c85aa0408a5bb9 current change 25420 trying: earliest ^51db83df9d588010d0bd995641c85aa0408a5bb9 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^51db83df9d588010d0bd995641c85aa0408a5bb9 Reading pipe: git cat-file commit e8f83909ceb570f5a7e48c2853f3c5d8207cea52 current change 25448 trying: earliest ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 Reading pipe: git cat-file commit 09a48eb7acd594dce52e06681be9c366e1844d66 current change 25521 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 ... """ The fix is two-fold: * detect an infinite loop and die right away instead of looping forever; * make sure, `git rev-list --bisect` can't return "latestCommit" again by excluding it from the rev-list range explicitly. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-05Merge branch 'ld/git-p4-shelve-update-fix'Libravatar Junio C Hamano1-0/+2
"git p4" failed to update a shelved change when there were moved files, which has been corrected. * ld/git-p4-shelve-update-fix: git-p4: handle update of moved/copied files when updating a shelve git-p4: add failing test for shelved CL update involving move/copy
2019-01-18git-p4: handle update of moved/copied files when updating a shelveLibravatar Luke Diamand1-0/+2
Perforce requires a complete list of files being operated on. If git is updating an existing shelved changelist, then any files which are moved or copied were not being added to this list. Signed-off-by: Luke Diamand <luke@diamand.org> Acked-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-07git-p4: fix problem when p4 login is not necessaryLibravatar Peter Osterlund1-0/+2
In a perforce setup where login is not required, communication fails because p4_check_access does not understand the response from the p4 client. Fixed by detecting and ignoring the "info" response. Signed-off-by: Peter Osterlund <peterosterlund2@gmail.com> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16git-p4: fully support unshelving changelistsLibravatar Luke Diamand1-36/+48
The previous git-p4 unshelve support would check for changes in Perforce to the files being unshelved since the original shelve, and would complain if any were found. This was to ensure that the user wouldn't end up with both the shelved change delta, and some deltas from other changes in their git commit. e.g. given fileA: the quick brown fox change1: s/the/The/ <- p4 shelve this change change2: s/fox/Fox/ <- p4 submit this change git p4 unshelve 1 <- FAIL This change teaches the P4Unshelve class to always create a parent commit which matches the P4 tree (for the files being unshelved) at the point prior to the P4 shelve being created (which is reported in the p4 description for a shelved changelist). That then means git-p4 can always create a git commit matching the P4 shelve that was originally created, without any extra deltas. The user might still need to use the --origin option though - there is no way for git-p4 to work out the versions of all of the other *unchanged* files in the shelve, since this information is not recorded by Perforce. Additionally this fixes handling of shelved 'move' operations. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16git-p4: unshelve into refs/remotes/p4-unshelved, not refs/remotes/p4/unshelvedLibravatar Luke Diamand1-1/+2
The branch detection code looks for branches under refs/remotes/p4/... and can end up getting confused if there are unshelved changes in there as well. This happens in the function p4BranchesInGit(). Instead, put the unshelved changes into refs/remotes/p4-unshelved/<N>. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-12git-p4: do not fail in verbose mode for missing 'fileSize' keyLibravatar Luke Diamand1-1/+4
If deleting or moving a file, sometimes P4 doesn't report the file size. The code handles this just fine but some logging crashes. Stop this happening. There was some earlier discussion on the list about this: https://public-inbox.org/git/xmqq1sqpp1vv.fsf@gitster.mtv.corp.google.com/ Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01git-p4: add the `p4-pre-submit` hookLibravatar Chen Bin1-1/+15
The `p4-pre-submit` hook is executed before git-p4 submits code. If the hook exits with non-zero value, submit process not start. Signed-off-by: Chen Bin <chenbin.sh@gmail.com> Reviewed-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19git-p4: python3: fix octal constantsLibravatar Luke Diamand1-1/+1
See PEP3127. Works fine with python2 as well. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19git-p4: python3: use print() functionLibravatar Luke Diamand1-124/+124
Replace calls to print ... with the function form, print(...), to allow use with python3 as well as python2.x. Converted using 2to3 (and some hand-editing). Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19git-p4: python3: basestring workaroundLibravatar Luke Diamand1-0/+16
In Python3, basestring no longer exists, so use this workaround. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19git-p4: python3: remove backticksLibravatar Luke Diamand1-1/+1
Backticks around a variable are a deprecated alias for repr(). This has been removed in python3, so just use the string representation instead, which is equivalent. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19git-p4: python3: replace dict.has_key(k) with "k in dict"Libravatar Luke Diamand1-39/+39
Python3 does not have the dict.has_key() function, so replace all such calls with "k in dict". This will still work with python2.6 and python2.7. Converted using 2to3 (plus some hand-editing) Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19git-p4: python3: replace <> with !=Libravatar Luke Diamand1-1/+1
The <> string inequality operator (which doesn't seem to be even documented) no longer exists in python3. Replace with !=. This still works with python2. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-18Merge branch 'ld/git-p4-updates'Libravatar Junio C Hamano1-24/+156
"git p4" updates. * ld/git-p4-updates: git-p4: auto-size the block git-p4: narrow the scope of exceptions caught when parsing an int git-p4: raise exceptions from p4CmdList based on error from p4 server git-p4: better error reporting when p4 fails git-p4: add option to disable syncing of p4/master with p4 git-p4: disable-rebase: allow setting this via configuration git-p4: add options --commit and --disable-rebase
2018-06-12git-p4: auto-size the blockLibravatar Luke Diamand1-6/+21
git-p4 originally would fetch changes in one query. On large repos this could fail because of the limits that Perforce imposes on the number of items returned and the number of queries in the database. To fix this, git-p4 learned to query changes in blocks of 512 changes, However, this can be very slow - if you have a few million changes, with each chunk taking about a second, it can be an hour or so. Although it's possible to tune this value manually with the "--changes-block-size" option, it's far from obvious to ordinary users that this is what needs doing. This change alters the block size dynamically by looking for the specific error messages returned from the Perforce server, and reducing the block size if the error is seen, either to the limit reported by the server, or to half the current block size. That means we can start out with a very large block size, and then let it automatically drop down to a value that works without error, while still failing correctly if some other error occurs. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>