summaryrefslogtreecommitdiff
path: root/t/t9818-git-p4-block.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-06-12git-p4: auto-size the blockLibravatar Luke Diamand1-0/+8
git-p4 originally would fetch changes in one query. On large repos this could fail because of the limits that Perforce imposes on the number of items returned and the number of queries in the database. To fix this, git-p4 learned to query changes in blocks of 512 changes, However, this can be very slow - if you have a few million changes, with each chunk taking about a second, it can be an hour or so. Although it's possible to tune this value manually with the "--changes-block-size" option, it's far from obvious to ordinary users that this is what needs doing. This change alters the block size dynamically by looking for the specific error messages returned from the Perforce server, and reducing the block size if the error is seen, either to the limit reported by the server, or to half the current block size. That means we can start out with a very large block size, and then let it automatically drop down to a value that works without error, while still failing correctly if some other error occurs. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-21git-p4: reduce number of server queries for fetchesLibravatar Sam Hocevar1-1/+1
When fetching changes from a depot using a full client spec, there is no need to perform as many queries as there are top-level paths in the client spec. Instead we query all changes in chronological order, also getting rid of the need to sort the results and remove duplicates. Signed-off-by: Sam Hocevar <sam@hocevar.net> Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-14git-p4: failing test case for skipping changes with multiple depotsLibravatar Luke Diamand1-1/+27
James Farwell reported that with multiple depots git-p4 would skip changes. http://article.gmane.org/gmane.comp.version-control.git/282297 Add a failing test case demonstrating the problem. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-10git-p4: fixing --changes-block-size handlingLibravatar Luke Diamand1-6/+6
The --changes-block-size handling was intended to help when a user has a limited "maxscanrows" (see "p4 group"). It used "p4 changes -m $maxchanges" to limit the number of results. Unfortunately, it turns out that the "maxscanrows" and "maxresults" limits are actually applied *before* the "-m maxchanges" parameter is considered (experimentally). Fix the block-size handling so that it gets blocks of changes limited by revision number ($Start..$Start+$N, etc). This limits the number of results early enough that both sets of tests pass. Note that many other Perforce operations can fail for the same reason (p4 print, p4 files, etc) and it's probably not possible to workaround this. In the real world, this is probably not usually a problem. Signed-off-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-08git-p4: test with limited p4 server resultsLibravatar Luke Diamand1-6/+23
Change the --changes-block-size git-p4 test to use an account with limited "maxresults" and "maxscanrows" values. These conditions are applied in the server *before* the "-m maxchanges" parameter to "p4 changes" is applied, and so the strategy that git-p4 uses for limiting the number of changes does not work. As a result, the tests all fail. Note that "maxscanrows" is set quite high, as it appears to not only limit results from "p4 changes", but *also* limits results from "p4 print". Files that have more than "maxscanrows" changes seem (experimentally) to be impossible to print. There's no good way to work around this. Signed-off-by: Luke Diamand <luke@diamand.org> Acked-by: Lex Spoon <lex@lexspoon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-08git-p4: additional testing of --changes-block-sizeLibravatar Luke Diamand1-9/+47
Add additional tests of some corner-cases of the --changes-block-size git-p4 parameter. Also reduce the number of p4 changes created during the tests, so that they complete faster. Signed-off-by: Luke Diamand <luke@diamand.org> Acked-by: Lex Spoon <lex@lexspoon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20git-p4: use -m when running p4 changesLibravatar Lex Spoon1-0/+64
Simply running "p4 changes" on a large branch can result in a "too many rows scanned" error from the Perforce server. It is better to use a sequence of smaller calls to "p4 changes", using the "-m" option to limit the size of each call. Signed-off-by: Lex Spoon <lex@lexspoon.org> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>