summaryrefslogtreecommitdiff
path: root/git-p4.py
AgeCommit message (Collapse)AuthorFilesLines
2020-06-02Merge branch 'bk/p4-prepare-p4-only-fix'Libravatar Junio C Hamano1-4/+5
The "--prepare-p4-only" option is supposed to stop after replaying one changeset, but kept going (by mistake?) * bk/p4-prepare-p4-only-fix: git-p4.py: fix --prepare-p4-only error with multiple commits
2020-05-12git-p4.py: fix --prepare-p4-only error with multiple commitsLibravatar Ben Keene1-4/+5
When using git p4 submit with the --prepare-p4-only option, the program should prepare a single p4 changelist and notify the user that more commits are pending and then stop processing. A bug has been introduced by the p4-changelist hook feature that causes the program to continue to try and process all pending changelists at the same time. The function applyCommit returns True when applying the commit was successful and the program should continue. However, when the optional flag --prepare-p4-only is set, the program should stop after the first application. Change the logic in the run method for P4Submit to check for the flag --prepare-p4-only after successfully completing the applyCommit method. Be aware - this change will fix the existing test error in t9807.23 for --prepare-p4-only. However there is insufficent coverage for this flag. If more than 1 commit is pending submission to P4, the method will properly prepare the P4 changelist, however it will still exit the application with an exitcode of 1. The current documentation does not define what the exit code should be in this condition. (See: https://git-scm.com/docs/git-p4#Documentation/git-p4.txt---prepare-p4-only) Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-10git-p4: recover from inconsistent perforce historyLibravatar Andrew Oakley1-2/+41
Perforce allows you commit files and directories with the same name, so you could have files //depot/foo and //depot/foo/bar both checked in. A p4 sync of a repository in this state fails. Deleting one of the files recovers the repository. When this happens we want git-p4 to recover in the same way as perforce. Note that Perforce has this change in their 2017.1 version: Bugs fixed in 2017.1 #1489051 (Job #2170) ** Submitting a file with the same name as an existing depot directory path (or vice versa) will now be rejected. so people hopefully will not creating damaged Perforce repos anymore, but "git p4" needs to be able to interact with already corrupt ones. Signed-off-by: Andrew Oakley <andrew@adoakley.name> Reviewed-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-22Merge branch 'bk/p4-pre-edit-changelist'Libravatar Junio C Hamano1-55/+181
"git p4" learned four new hooks and also "--no-verify" option to bypass them (and the existing "p4-pre-submit" hook). * bk/p4-pre-edit-changelist: git-p4: add RCS keyword status message git-p4: add p4 submit hooks git-p4: restructure code in submit git-p4: add --no-verify option git-p4: add p4-pre-submit exit text git-p4: create new function run_git_hook git-p4: rewrite prompt to be Windows compatible
2020-03-25Merge branch 'yz/p4-py3'Libravatar Junio C Hamano1-95/+140
Update "git p4" to work with Python 3. * yz/p4-py3: ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere git-p4: use python3's input() everywhere git-p4: simplify regex pattern generation for parsing diff-tree git-p4: use dict.items() iteration for python3 compatibility git-p4: use functools.reduce instead of reduce git-p4: fix freezing while waiting for fast-import progress git-p4: use marshal format version 2 when sending to p4 git-p4: open .gitp4-usercache.txt in text mode git-p4: convert path to unicode before processing them git-p4: encode/decode communication with git for python3 git-p4: encode/decode communication with p4 for python3 git-p4: remove string type aliasing git-p4: change the expansion test from basestring to list git-p4: make python2.7 the oldest supported version
2020-02-14git-p4: add RCS keyword status messageLibravatar Ben Keene1-0/+4
During the p4 submit process, git-p4 will attempt to apply a patch to the files found in the p4 workspace. However, if P4 uses RCS keyword expansion, this patch may fail. When the patch fails, the user is alerted to the failure and that git-p4 will attempt to clear the expanded text from the files and re-apply the patch. The current version of git-p4 does not tell the user the result of the re-apply attempt after the RCS expansion has been removed which can be confusing. Add a new print statement after the git patch has been successfully applied when the RCS keywords have been cleansed. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-14git-p4: add p4 submit hooksLibravatar Ben Keene1-1/+34
The git command "commit" supports a number of hooks that support changing the behavior of the commit command. The git-p4.py program only has one existing hook, "p4-pre-submit". This command occurs early in the process. There are no hooks in the process flow for modifying the P4 changelist text programmatically. Adds 3 new hooks to git-p4.py to the submit option. The new hooks are: * p4-prepare-changelist - Execute this hook after the changelist file has been created. The hook will be executed even if the --prepare-p4-only option is set. This hook ignores the --no-verify option in keeping with the existing behavior of git commit. * p4-changelist - Execute this hook after the user has edited the changelist. Do not execute this hook if the user has selected the --prepare-p4-only option. This hook will honor the --no-verify, following the conventions of git commit. * p4-post-changelist - Execute this hook after the P4 submission process has completed successfully. This hook takes no parameters and is executed regardless of the --no-verify option. It's return value will not be checked. The calls to the new hooks: p4-prepare-changelist, p4-changelist, and p4-post-changelist should all be called inside the try-finally block. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-14git-p4: restructure code in submitLibravatar Ben Keene1-41/+51
In preparation for adding new hooks to the submit method of git-p4, restructure the applyCommit function in the P4Submit class. Make the following changes: * Move all the code after the definition of submitted = False into the Try-Finally block. This ensures that any error that occurs will properly recover. This is not necessary with the current code because none of it should throw an exception, however the next set of changes will introduce exceptional code. Existing flow control can remain as defined - the if-block for prepare-p4-only sets the local variable "submitted" to True and exits the function. New early exits, leave submitted set to False so the Finally block will undo changes to the P4 workspace. * Make the small usability change of adding an empty string to the print statements displayed to the user when the prepare-p4-only option is selected. On Windows, the command print() may display a set of parentheses "()" to the user when the print() function is called with no parameters. By supplying an empty string, the intended blank line will print as expected. * Fix a small bug when the submittedTemplate is edited by the user and all content in the file is removed. The existing code will throw an exception if the separateLine is not found in the file. Change this code to test for the separator line using a find() test first and only split on the separator if it is found. * Additionally, add the new behavior that if the changelist file has been completely emptied that the Submit action for this changelist will be aborted. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-11git-p4: add --no-verify optionLibravatar Ben Keene1-13/+19
Add new command line option --no-verify: Add a new command line option "--no-verify" to the Submit command of git-p4.py. This option will function in the spirit of the existing --no-verify command line option found in git commit. It will cause the P4 Submit function to ignore the existing p4-pre-submit. Change the execution of the existing trigger p4-pre-submit to honor the --no-verify option. Before exiting on failure of this hook, display text to the user explaining which hook has failed and the impact of using the --no-verify option. Change the call of the p4-pre-submit hook to use the new run_git_hook function. This is in preparation of additional hooks to be added. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-11git-p4: add p4-pre-submit exit textLibravatar Ben Keene1-1/+9
When the p4-pre-submit exits with a non-zero exit code, the application will abort the process with no additional information presented to the user. This can be confusing for new users as it may not be clear that the p4-pre-submit action caused the error. Add text to explain why the program aborted the submit action. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-11git-p4: create new function run_git_hookLibravatar Ben Keene1-7/+69
This commit is in preparation of introducing new p4 submit hooks. The current code in the python script git-p4.py makes the assumption that the git hooks can be executed by subprocess.call() function. However, when git is run on Windows, this may not work as expected. The subprocess.call() does not cover all the use cases for properly executing the various types of executable files on Windows. Prepare for remediation by adding a new function, run_git_hook, that takes 2 parameters: * the short filename of an optionally registered git hook * an optional list of parameters The run_git_hook function will honor the existing behavior seen in the current code for executing the p4-pre-submit hook: * Hooks are looked for in core.hooksPath directory. * If core.hooksPath is not set, then the current .git/hooks directory is checked. * If the hook does not exist, the function returns True. * If the hook file is not accessible, the function returns True. * If the hook returns a zero exit code when executed, the function return True. * If the hook returns a non-zero exit code, the function returns False. Add the following additional functionality if git-p4.py is run on Windows. * If hook file is not located without an extension, search for any file in the associated hook directory (from the list above) that has the same name but with an extension. * If the file is still not found, return True (the hook is missing) Add a new function run_hook_command() that wraps the OS dependent functionality for actually running the subprocess.call() with OS dependent behavior: If a hook file exists on Windows: * If there is no extension, set the launch executable to be SH.EXE - Look for SH.EXE under the environmental variable EXEPATH in the bin/ directory. - If %EXEPATH%/bin/sh.exe exists, use this as the actual executable. - If %EXEPATH%/bin/sh.exe does not exist, use sh.exe - Execute subprocess.call() without the shell (shell=False) * If there is an extension, execute subprocess.call() with teh shell (shell=True) and consider the file to be the executable. The return value from run_hook_command() is the subprocess.call() return value. These functions are added in this commit, but are only staged and not yet used. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-11git-p4: rewrite prompt to be Windows compatibleLibravatar Ben Keene1-1/+4
The existing function prompt(prompt_text) does not work correctly when run on Windows 10 bash terminal when launched from the sourcetree GUI application. The stdout is not flushed properly so the prompt text is not displayed to the user until the next flush of stdout, which is quite confusing. Change this method by: * Adding flush to stderr, stdout, and stdin * Use readline from sys.stdin instead of raw_input. The existing strip().lower() are retained. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: avoid leak of file handle when cloningLibravatar Luke Diamand1-1/+2
Spotted by Eric Sunshine: https://public-inbox.org/git/CAPig+cRx3hG64nuDie69o_gdX39F=sR6D8LyA7J1rCErgu0aMA@mail.gmail.com/ Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: check for access to remote host earlierLibravatar Luke Diamand1-0/+2
Check we can talk to the remote host before starting the git-fastimport subchild. Otherwise we fail to connect, and then exit, leaving git-fastimport still running since we did not wait() for it. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: cleanup better on error exitLibravatar Luke Diamand1-15/+28
After an error, git-p4 calls die(). This just exits, and leaves child processes still running. Instead of calling die(), raise an exception and catch it where the child process(es) (git-fastimport) are created. This was analyzed in detail here: https://public-inbox.org/git/20190227094926.GE19739@szeder.dev/ This change does not address the particular issue of p4CmdList() invoking a subchild and not waiting for it on error. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: create helper function importRevisions()Libravatar Luke Diamand1-64/+68
This makes it easier to try/catch around this block of code to ensure cleanup following p4 failures is handled properly. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: disable some pylint warnings, to get pylint output to something ↵Libravatar Luke Diamand1-0/+8
manageable pylint is incredibly useful for finding bugs, but git-p4 has never used it, so there are a lot of warnings that while important, don't actually result in bugs. Let's turn those off for now, so we can get some useful output. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: add P4CommandException to report errors talking to PerforceLibravatar Luke Diamand1-0/+8
Currently when there is a P4 error, git-p4 calls die() which just exits. This then leaves the git-fast-import process still running, and can even leave p4 itself still running. As a result, git-p4 fails to exit cleanly. This is a particular problem for people running the unit tests in regression. Use this exception to report errors upwards, cleaning up as the error propagates. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30git-p4: make closeStreams() idempotentLibravatar Luke Diamand1-0/+3
Ensure that we can safely call self.closeStreams() multiple times, and can also call it even if there is no git fast-import stream at all. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use python3's input() everywhereLibravatar Yang Zhao1-2/+12
Python3 deprecates raw_input() from 2.7 and replaced it with input(). Since we do not need 2.7's input() semantics, `raw_input()` is aliased to `input()` for easy forward compatability. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: simplify regex pattern generation for parsing diff-treeLibravatar Yang Zhao1-7/+6
It is not clear why a generator was used to create the regex used to parse git-diff-tree output; I assume an early implementation required it, but is not part of the mainline change. Simply use a lazily initialized global instead. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use dict.items() iteration for python3 compatibilityLibravatar Yang Zhao1-1/+1
Python3 uses dict.items() instead of .iteritems() to provide iteratoration over dict. Although items() is technically less efficient for python2.7 (allocates a new list instead of simply iterating), the amount of data involved is very small and the penalty negligible. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use functools.reduce instead of reduceLibravatar Yang Zhao1-1/+2
For python3, reduce() has been moved to functools.reduce(). This is also available in python2.7. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: fix freezing while waiting for fast-import progressLibravatar Yang Zhao1-0/+1
As part of its importing process, git-p4 sends a `checkpoint` followed immediately by `progress` to fast-import to force synchronization. Due to buffering, it is possible for the `progress` command to not be flushed before git-p4 begins to wait for the corresponding response. This causes the script to freeze completely, and is consistently observable at least on python-3.6.9. Make sure this command sequence is completely flushed before waiting. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: use marshal format version 2 when sending to p4Libravatar Yang Zhao1-1/+2
p4 does not appear to understand marshal format version 3 and above. Version 2 was the latest supported by python-2.7. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: open .gitp4-usercache.txt in text modeLibravatar Yang Zhao1-2/+2
Opening .gitp4-usercache.txt in text mode makes python 3 happy without explicitly adding encoding and decoding. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: convert path to unicode before processing themLibravatar Yang Zhao1-25/+44
P4 allows essentially arbitrary encoding for path data while we would perfer to be dealing only with unicode strings. Since path data need to survive round-trip back to p4, this patch implements the general policy that we store path data as-is, but decode them to unicode before doing any non-trivial processing. A new `decode_path()` method is provided that generally does the correct conversion, taking into account `git-p4.pathEncoding` configuration. For python2.7, path strings will be left as-is if it only contains ASCII characters. For python3, decoding is always done so that we have str objects. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: encode/decode communication with git for python3Libravatar Yang Zhao1-4/+17
Under python3, calls to write() on the stream to `git fast-import` must be encoded. This patch wraps the IO object such that this encoding is done transparently. Conversely, any text data read from subprocesses must also be decoded before running through the rest of the pipeline. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Reviewed-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: encode/decode communication with p4 for python3Libravatar Yang Zhao1-13/+46
The marshalled dict in the response given on STDOUT by p4 uses `str` for keys and string values. When run using python3, these values are deserialized as `bytes`, leading to a whole host of problems as the rest of the code assumes `str` is used throughout. This patch changes the deserialization behaviour such that, as much as possible, text output from p4 is decoded to native unicode strings. Exceptions are made for the field `data` as it is usually arbitrary binary data. `depotFile[0-9]*`, `path`, and `clientFile` are also exempt as they contain path strings not encoded with UTF-8, and must survive round-trip back to p4. Conversely, text data being piped to p4 must always be encoded when running under python3. encode_text_stream() and decode_text_stream() were added to make these transformations more convenient. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: remove string type aliasingLibravatar Yang Zhao1-16/+0
Now that python2.7 is the minimum required version and we no longer use the basestring type, it is not necessary to use type aliasing to ensure python3 compatibility. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: change the expansion test from basestring to listLibravatar Ben Keene1-9/+9
Python 3 handles strings differently than Python 2.7. Since Python 2 is reaching it's end of life, a series of changes are being submitted to enable python 3.5 and following support. The current code fails basic tests under python 3.5. Some codepaths can represent a command line the program internally prepares to execute either as a single string (i.e. each token properly quoted, concatenated with $IFS) or as a list of argv[] elements, and there are 9 places where we say "if X is isinstance(_, basestring), then do this thing to handle X as a command line in a single string; if not, X is a command line in a list form". This does not work well with Python 3, as there is no basestring (everything is Unicode now), and even with Python 2, it was not an ideal way to tell the two cases apart, because an internally formed command line could have been in a single Unicode string. Flip the check to say "if X is not a list, then handle X as a command line in a single string; otherwise treat it as a command line in a list form". This will get rid of references to 'basestring', to migrate the code ready for Python 3. Thanks-to: Junio C Hamano <gitster@pobox.com> Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15git-p4: make python2.7 the oldest supported versionLibravatar Yang Zhao1-18/+2
Python2.6 and earlier have been end-of-life'd for many years now, and we actually already use 2.7-only features in the code. Make the version check reflect current realities. This also removes the need to explicitly define CalledProcessError if it's not available. Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-02Merge branch 'bk/p4-misc-usability'Libravatar Junio C Hamano1-32/+42
Miscellaneous small UX improvements on "git-p4". * bk/p4-misc-usability: git-p4: show detailed help when parsing options fail git-p4: yes/no prompts should sanitize user text
2019-12-16git-p4: show detailed help when parsing options failLibravatar Ben Keene1-1/+6
When a user provides invalid parameters to git-p4, the program reports the failure but does not provide the correct command syntax. Add an exception handler to the command-line argument parser to display the command's specific command line parameter syntax when an exception is thrown. Rethrow the exception so the current behavior is retained. Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-16git-p4: yes/no prompts should sanitize user textLibravatar Ben Keene1-31/+36
When prompting the user interactively for direction, the tests are not forgiving of user input format. For example, the first query asks for a yes/no response. If the user enters the full word "yes" or "no" or enters a capital "Y" the test will fail. Create a new function, prompt(prompt_text) where * prompt_text is the text prompt for the user * returns a single character where valid return values are found by inspecting prompt_text for single characters surrounded by square brackets This new function must prompt the user for input and sanitize it by converting the response to a lower case string, trimming leading and trailing spaces, and checking if the first character is in the list of choices. If it is, return the first letter. Change the current references to raw_input() to use this new function. Since the method requires the returned text to be one of the available choices, remove the loop from the calling code that handles response verification. Thanks-to: Denton Liu <liu.denton@gmail.com> Signed-off-by: Ben Keene <seraphire@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-11git-p4: honor lfs.storage configuration variableLibravatar r.burenkov1-2/+8
"git lfs" allows users to specify the custom storage location with the configuration variable `lfs.storage`, but when interacting with GitLFS pointers, "git p4" always uses the hardcoded default that is the `.git/lfs/` directory, without paying attention to the configuration. Use the value configured in `lfs.storage`, if exists, as all the "git" operations do, for consistency. Signed-off-by: r.burenkov <panzercheg@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-06git-p4: auto-delete named temporary fileLibravatar Philip.McGraw1-7/+6
Avoid double-open exceptions on Windows platform when calculating for lfs compressed size threshold (git-p4.largeFileCompressedThreshold) comparisons. Take new approach using the NamedTemporaryFile() file-like object as input to the ZipFile() which auto-deletes after implicit close leaving with scope. Original code had double-open exception on Windows platform because file still open from NamedTemporaryFile() using generated filename instead of object. Thanks to Andrey for patiently suggesting several iterations on this change for avoiding exceptions! Also print error details after resulting IOError to make debugging cause of exception less mysterious when it has nothing to do with "git version recent enough." Signed-off-by: Philip.McGraw <Philip.McGraw@bentley.com> Reviewed-by: Andrey Mazo <ahippo@yandex.com> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-09Merge branch 'am/p4-branches-excludes'Libravatar Junio C Hamano1-17/+27
"git p4" update. * am/p4-branches-excludes: git-p4: respect excluded paths when detecting branches git-p4: add failing test for "git-p4: respect excluded paths when detecting branches" git-p4: don't exclude other files with same prefix git-p4: add failing test for "don't exclude other files with same prefix" git-p4: don't groom exclude path list on every commit git-p4: match branches case insensitively if configured git-p4: add failing test for "git-p4: match branches case insensitively if configured" git-p4: detect/prevent infinite loop in gitCommitByP4Change()
2019-06-17Merge branch 'mm/p4-unshelve-windows-fix'Libravatar Junio C Hamano1-1/+1
The command line to invoke a "git cat-file" command from inside "git p4" was not properly quoted to protect a caret and running a broken command on Windows, which has been corrected. * mm/p4-unshelve-windows-fix: p4 unshelve: fix "Not a valid object name HEAD0" on Windows
2019-05-28p4 unshelve: fix "Not a valid object name HEAD0" on WindowsLibravatar Mike Mueller1-1/+1
git p4 unshelve was failing with these errors: fatal: Not a valid object name HEAD0 Command failed: git cat-file commit HEAD^0 (git version 2.21.0.windows.1, python 2.7.16) The pOpen call used by git-p4 to invoke the git command can take either a string or an array as a first argument. The array form is preferred because platform-specific escaping of special characters will be handled automatically.(https://docs.python.org/2/library/subprocess.html) The extractLogMessageFromGitCommit method was, however, using the string form and so the caret (^) character in the HEAD^0 argument was not being escaped on Windows. The caret happens to be the escape character, which is why the git command was receiving HEAD0. The behaviour can be confirmed by typing ECHO HEAD^0 at the command- prompt, which emits HEAD0. The solution is simply to use the array format of passing the command to fOpen, which is recommended and used in other parts of this code anyway. Signed-off-by: Mike Mueller <mike.mueller@moodys.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-28git-p4: allow unshelving of branched filesLibravatar Simon Williams1-1/+1
When unshelving a changelist, git-p4 tries to work out the appropriate parent commit in a given branch (default: HEAD). To do this, it looks at the state of any pre-existing files in the target Perforce branch, omitting files added in the shelved changelist. Currently, only files added (or move targets) are classed as new. However, files integrated from other branches (i.e. a 'branch' action) also need to be considered as added, for this purpose. Signed-off-by: Simon Williams <simon@no-dns-yet.org.uk> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: respect excluded paths when detecting branchesLibravatar Mazo, Andrey1-2/+1
Currently, excluded paths are only handled in the following cases: * no branch detection; * branch detection with using clientspec. However, excluded paths are not respected in case of branch detection without using clientspec. Fix this by consulting the list of excluded paths when splitting files across branches. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: don't exclude other files with same prefixLibravatar Mazo, Andrey1-7/+14
Make sure not to exclude files unintentionally if exclude paths are specified without a trailing /. I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file". Do this by ensuring that paths without a trailing "/" are only matched completely. Also, abort path search on the first match as a micro-optimization. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: don't groom exclude path list on every commitLibravatar Mazo, Andrey1-5/+7
Currently, `cloneExclude` array is being groomed (by removing trailing "...") on every changeset. (since `extractFilesFromCommit()` is called on every imported changeset) As a micro-optimization, do it once while parsing arguments. Also, prepend "/" and remove trailing "..." at the same time. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: match branches case insensitively if configuredLibravatar Mazo, Andrey1-2/+2
git-p4 knows how to handle case insensitivity in file paths if core.ignorecase is set. However, when determining a branch for a file, it still does a case-sensitive prefix match. This may result in some file changes to be lost on import. For example, given the following commits 1. add //depot/main/file1 2. add //depot/DirA/file2 3. add //depot/dira/file3 4. add //depot/DirA/file4 and "branchList = main:DirA" branch mapping, commit 3 will be lost. So, do branch search case insensitively if running with core.ignorecase set. Teach splitFilesIntoBranches() to use the p4PathStartsWith() function for path prefix matches instead of always case-sensitive match. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02git-p4: detect/prevent infinite loop in gitCommitByP4Change()Libravatar Mazo, Andrey1-1/+3
Under certain circumstances, gitCommitByP4Change() can enter an infinite loop resulting in `git p4 sync` hanging forever. The problem is that `git rev-list --bisect <latest> ^<earliest>` can return `<latest>`, which would result in reinspecting <latest> and potentially an infinite loop. This can happen when importing just a subset of P4 repository and/or with explicit "--changesfile" option. A real-life example: """ looking in ref refs/remotes/p4/mybranch for change 26894 using bisect... Reading pipe: git rev-parse refs/remotes/p4/mybranch trying: earliest latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git cat-file commit 147f5d3292af2e1cc4a56a7b96db845144c68486 current change 25339 trying: earliest ^147f5d3292af2e1cc4a56a7b96db845144c68486 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^147f5d3292af2e1cc4a56a7b96db845144c68486 Reading pipe: git cat-file commit 51db83df9d588010d0bd995641c85aa0408a5bb9 current change 25420 trying: earliest ^51db83df9d588010d0bd995641c85aa0408a5bb9 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^51db83df9d588010d0bd995641c85aa0408a5bb9 Reading pipe: git cat-file commit e8f83909ceb570f5a7e48c2853f3c5d8207cea52 current change 25448 trying: earliest ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 Reading pipe: git cat-file commit 09a48eb7acd594dce52e06681be9c366e1844d66 current change 25521 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 ... """ The fix is two-fold: * detect an infinite loop and die right away instead of looping forever; * make sure, `git rev-list --bisect` can't return "latestCommit" again by excluding it from the rev-list range explicitly. Signed-off-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-05Merge branch 'ld/git-p4-shelve-update-fix'Libravatar Junio C Hamano1-0/+2
"git p4" failed to update a shelved change when there were moved files, which has been corrected. * ld/git-p4-shelve-update-fix: git-p4: handle update of moved/copied files when updating a shelve git-p4: add failing test for shelved CL update involving move/copy
2019-01-18git-p4: handle update of moved/copied files when updating a shelveLibravatar Luke Diamand1-0/+2
Perforce requires a complete list of files being operated on. If git is updating an existing shelved changelist, then any files which are moved or copied were not being added to this list. Signed-off-by: Luke Diamand <luke@diamand.org> Acked-by: Andrey Mazo <amazo@checkvideo.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-07git-p4: fix problem when p4 login is not necessaryLibravatar Peter Osterlund1-0/+2
In a perforce setup where login is not required, communication fails because p4_check_access does not understand the response from the p4 client. Fixed by detecting and ignoring the "info" response. Signed-off-by: Peter Osterlund <peterosterlund2@gmail.com> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16git-p4: fully support unshelving changelistsLibravatar Luke Diamand1-36/+48
The previous git-p4 unshelve support would check for changes in Perforce to the files being unshelved since the original shelve, and would complain if any were found. This was to ensure that the user wouldn't end up with both the shelved change delta, and some deltas from other changes in their git commit. e.g. given fileA: the quick brown fox change1: s/the/The/ <- p4 shelve this change change2: s/fox/Fox/ <- p4 submit this change git p4 unshelve 1 <- FAIL This change teaches the P4Unshelve class to always create a parent commit which matches the P4 tree (for the files being unshelved) at the point prior to the P4 shelve being created (which is reported in the p4 description for a shelved changelist). That then means git-p4 can always create a git commit matching the P4 shelve that was originally created, without any extra deltas. The user might still need to use the --origin option though - there is no way for git-p4 to work out the versions of all of the other *unchanged* files in the shelve, since this information is not recorded by Perforce. Additionally this fixes handling of shelved 'move' operations. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>