summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2019-06-19commit-graph: merge commit-graph chainsLibravatar Derrick Stolee3-33/+240
When searching for a commit in a commit-graph chain of G graphs with N commits, the search takes O(G log N) time. If we always add a new tip graph with every write, the linear G term will start to dominate and slow the lookup process. To keep lookups fast, but also keep most incremental writes fast, create a strategy for merging levels of the commit-graph chain. The strategy is detailed in the commit-graph design document, but is summarized by these two conditions: 1. If the number of commits we are adding is more than half the number of commits in the graph below, then merge with that graph. 2. If we are writing more than 64,000 commits into a single graph, then merge with all lower graphs. The numeric values in the conditions above are currently constant, but can become config options in a future update. As we merge levels of the commit-graph chain, check that the commits still exist in the repository. A garbage-collection operation may have removed those commits from the object store and we do not want to persist them in the commit-graph chain. This is a non-issue if the 'git gc' process wrote a new, single-level commit-graph file. After we merge levels, the old graph-{hash}.graph files are no longer referenced by the commit-graph-chain file. We will expire these files in a future change. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: add --split option to builtinLibravatar Derrick Stolee3-8/+138
Add a new "--split" option to the 'git commit-graph write' subcommand. This option allows the optional behavior of writing a commit-graph chain. The current behavior will add a tip commit-graph containing any commits that are not in the existing commit-graph or commit-graph chain. Later changes will allow merging the chain and expiring out-dated files. Add a new test script (t5324-split-commit-graph.sh) that demonstrates this behavior. Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: write commit-graph chainsLibravatar Derrick Stolee3-12/+278
Extend write_commit_graph() to write a commit-graph chain when given the COMMIT_GRAPH_SPLIT flag. This implementation is purposefully simplistic in how it creates a new chain. The commits not already in the chain are added to a new tip commit-graph file. Much of the logic around writing a graph-{hash}.graph file and updating the commit-graph-chain file is the same as the commit-graph file case. However, there are several places where we need to do some extra logic in the split case. Track the list of graph filenames before and after the planned write. This will be more important when we start merging graph files, but it also allows us to upgrade our commit-graph file to the appropriate graph-{hash}.graph file when we upgrade to a chain of commit-graphs. Note that we use the eighth byte of the commit-graph header to store the number of base graph files. This determines the length of the base graphs chunk. A subtle change of behavior with the new logic is that we do not write a commit-graph if we our commit list is empty. This extends to the typical case, which is reflected in t5318-commit-graph.sh. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: rearrange chunk count logicLibravatar Derrick Stolee1-14/+21
The number of chunks in a commit-graph file can change depending on whether we need the Extra Edges Chunk. We are going to add more optional chunks, and it will be helpful to rearrange this logic around the chunk count before doing so. Specifically, we need to finalize the number of chunks before writing the commit-graph header. Further, we also need to fill out the chunk lookup table dynamically and using "num_chunks" as we add optional chunks is useful for adding optional chunks in the future. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: add base graphs chunkLibravatar Derrick Stolee3-2/+32
To quickly verify a commit-graph chain is valid on load, we will read from the new "Base Graphs Chunk" of each file in the chain. This will prevent accidentally loading incorrect data from manually editing the commit-graph-chain file or renaming graph-{hash}.graph files. The commit_graph struct already had an object_id struct "oid", but it was never initialized or used. Add a line to read the hash from the end of the commit-graph file and into the oid member. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: load commit-graph chainsLibravatar Derrick Stolee1-6/+106
Prepare the logic for reading a chain of commit-graphs. First, look for a file at $OBJDIR/info/commit-graph. If it exists, then use that file and stop. Next, look for the chain file at $OBJDIR/info/commit-graphs/commit-graph-chain. If this file exists, then load the hash values as line-separated values in that file and load $OBJDIR/info/commit-graphs/graph-{hash[i]}.graph for each hash[i] in that file. The file is given in order, so the first hash corresponds to the "base" file and the final hash corresponds to the "tip" file. This implementation assumes that all of the graph-{hash}.graph files are in the same object directory as the commit-graph-chain file. This will be updated in a future change. This change is purposefully simple so we can isolate the different concerns. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: rename commit_compare to oid_compareLibravatar Derrick Stolee1-2/+2
The helper function commit_compare() actually compares object_id structs, not commits. A future change to commit-graph.c will need to sort commit structs, so rename this function in advance. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: prepare for commit-graph chainsLibravatar Derrick Stolee2-11/+81
To prepare for a chain of commit-graph files, augment the commit_graph struct to point to a base commit_graph. As we load commits from the graph, we may actually want to read from a base file according to the graph position. The "graph position" of a commit is given by concatenating the lexicographic commit orders from each of the commit-graph files in the chain. This means that we must distinguish two values: * lexicographic index : the position within the lexicographic order in a single commit-graph file. * graph position: the position within the concatenated order of multiple commit-graph files Given the lexicographic index of a commit in a graph, we can compute the graph position by adding the number of commits in the lower-level graphs. To find the lexicographic index of a commit, we subtract the number of commits in lower-level graphs. While here, change insert_parent_or_die() to take a uint32_t position, as that is the type used by its only caller and that makes more sense with the limits in the commit-graph format. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19commit-graph: document commit-graph chainsLibravatar Derrick Stolee1-0/+59
Add a basic description of commit-graph chains. More details about the feature will be added as we add functionality. This introduction gives a high-level overview to the goals of the feature and the basic layout of commit-graph chains. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12packfile: rename close_all_packs to close_object_storeLibravatar Derrick Stolee11-12/+12
The close_all_packs() method is now responsible for more than just pack-files. It also closes the commit-graph and the multi-pack-index. Rename the function to be more descriptive of its larger role. The name also fits because the input parameter is a raw_object_store. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12packfile: close commit-graph in close_all_packsLibravatar Derrick Stolee1-0/+3
The close_all_packs() method is used to close all read handles to pack-files and the multi-pack-index before running 'git gc --auto'. This is particularly important on the Windows platform, where read handles block any writes to those files. Replacing one of these files with a rename() will fail in this situation. The commit-graph also performs a rename, so is susceptable to this problem. We are careful to close the commit-graph before writing, but that doesn't work when a 'git fetch' (or similar) process runs 'git gc --auto' which may write a commit-graph. Here, close the commit-graph as part of close_all_packs(). Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: use raw_object_store when closingLibravatar Derrick Stolee3-6/+6
The close_commit_graph() method took a repository struct, but then only uses the raw_object_store within. Change the function prototype to make the method more flexible. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: extract write_commit_graph_file()Libravatar Derrick Stolee1-75/+80
The write_commit_graph() method is too complex, so we are extracting helper functions one by one. Extract write_commit_graph_file() that takes all of the information in the context struct and writes the data to a commit-graph file. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: extract copy_oids_to_commits()Libravatar Derrick Stolee1-25/+32
The write_commit_graph() method is too complex, so we are extracting helper functions one by one. Extract copy_oids_to_commits(), which fills the commits list with the distinct commits from the oids list. During this loop, it also counts the number of "extra" edges from octopus merges. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: extract count_distinct_commits()Libravatar Derrick Stolee1-13/+22
The write_commit_graph() method is too complex, so we are extracting helper functions one by one. Extract count_distinct_commits(), which sorts the oids list, then iterates through to find duplicates. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: extract fill_oids_from_all_packs()Libravatar Derrick Stolee1-11/+15
The write_commit_graph() method is too complex, so we are extracting helper functions one by one. Extract fill_oids_from_all_packs() that reads all pack-files for commits and fills the oid list in the context. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: extract fill_oids_from_commit_hex()Libravatar Derrick Stolee1-32/+40
The write_commit_graph() method is too complex, so we are extracting helper functions one by one. Extract fill_oids_from_commit_hex() that reads the given commit id list and fille the oid list in the context. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: extract fill_oids_from_packs()Libravatar Derrick Stolee1-36/+47
The write_commit_graph() method is too complex, so we are extracting helper functions one by one. This extracts fill_oids_from_packs() that reads the given pack-file list and fills the oid list in the context. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: create write_commit_graph_contextLibravatar Derrick Stolee1-196/+194
The write_commit_graph() method is too large and complex. To simplify it, we should extract several helper functions. However, we will risk repeating a lot of declarations related to progress incidators and object id or commit lists. Create a new write_commit_graph_context struct that contains the core data structures used in this process. Replace the other local variables with the values inside the context object. Following this change, we will start to lift code segments wholesale out of the write_commit_graph() method and into helper functions. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: remove Future Work sectionLibravatar Derrick Stolee1-17/+0
The commit-graph feature began with a long list of planned benefits, most of which are now complete. The future work section has only a few items left. As for making more algorithms aware of generation numbers, some are only waiting for generation number v2 to ensure the performance matches the existing behavior using commit date. It is unlikely that we will ever send a commit-graph file as part of the protocol, since we would need to verify the data, and that is expensive. If we want to start trusting remote content, then that item can be investigated again. While there is more work to be done on the feature, having a section of the docs devoted to a TODO list is wasteful and hard to keep up-to-date. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: collapse parameters into flagsLibravatar Derrick Stolee5-13/+18
The write_commit_graph() and write_commit_graph_reachable() methods currently take two boolean parameters: 'append' and 'report_progress'. As we update these methods, adding more parameters this way becomes cluttered and hard to maintain. Collapse these parameters into a 'flags' parameter, and adjust the callers to provide flags as necessary. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12commit-graph: return with errors during writeLibravatar Derrick Stolee6-39/+77
The write_commit_graph() method uses die() to report failure and exit when confronted with an unexpected condition. This use of die() in a library function is incorrect and is now replaced by error() statements and an int return type. Return zero on success and a negative value on failure. Now that we use 'goto cleanup' to jump to the terminal condition on an error, we have new paths that could lead to uninitialized values. New initializers are added to correct for this. The builtins 'commit-graph', 'gc', and 'commit' call these methods, so update them to check the return value. Test that 'git commit-graph write' returns a proper error code when hitting a failure condition in write_commit_graph(). Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-13commit-graph: fix the_repository referenceLibravatar Derrick Stolee1-1/+1
The parse_commit_buffer() method takes a repository pointer, so it should not refer to the_repository anymore. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph: improve & i18n error messagesLibravatar Ævar Arnfjörð Bjarmason1-19/+19
Change the error emitted when a commit-graph file is corrupt so that we actually mention the commit-graph, e.g. change errors like: error: improper chunk offset 0000000000385e0c To: error: commit-graph improper chunk offset 0000000000385e0c As discussed in the commits leading up to this one the commit-graph machinery is now used by common commands like "status". If the graph was corrupt we'd often emit some error that gave no indication what was wrong. Now some of them are still cryptic, but they'll at least mention "commit-graph" to give the user a hint as to where to look. While I'm at it mark some of the strings that hadn't been marked for translation. It's clear from the commit history and the code that this was merely forgotten at the time, and wasn't intentional.p5 Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph write: don't die if the existing graph is corruptLibravatar Ævar Arnfjörð Bjarmason4-5/+23
When the commit-graph is written we end up calling parse_commit(). This will in turn invoke code that'll consult the existing commit-graph about the commit, if the graph is corrupted we die. We thus get into a state where a failing "commit-graph verify" can't be followed-up with a "commit-graph write" if core.commitGraph=true is set, the graph either needs to be manually removed to proceed, or core.commitGraph needs to be set to "false". Change the "commit-graph write" codepath to use a new parse_commit_no_graph() helper instead of parse_commit() to avoid this. The latter will call repo_parse_commit_internal() with use_commit_graph=1 as seen in 177722b344 ("commit: integrate commit graph with commit parsing", 2018-04-10). Not using the old graph at all slows down the writing of the new graph by some small amount, but is a sensible way to prevent an error in the existing commit-graph from spreading. Just fixing the current issue would be likely to result in code that's inadvertently broken in the future. New code might use the commit-graph at a distance. To detect such cases introduce a "GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD" setting used when we do our corruption tests, and test that a "write/verify" combo works after every one of our current test cases where we now detect commit-graph corruption. Some of the code changes here might be strictly unnecessary, e.g. I was unable to find cases where the parse_commit() called from write_graph_chunk_data() didn't exit early due to "item->object.parsed" being true in repo_parse_commit_internal() (before the use_commit_graph=1 has any effect). But let's also convert those cases for good measure, we do not have exhaustive tests for all possible types of commit-graph corruption. This might need to be re-visited if we learn to write the commit-graph incrementally, but probably not. Hopefully we'll just start by finding out what commits we have in total, then read the old graph(s) to see what they cover, and finally write a new graph file with everything that's missing. In that case the new graph writing code just needs to continue to use e.g. a parse_commit() that doesn't consult the existing commit-graphs. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph verify: detect inability to read the graphLibravatar Ævar Arnfjörð Bjarmason2-1/+9
Change "commit-graph verify" to error on open() failures other than ENOENT. As noted in the third paragraph of 283e68c72f ("commit-graph: add 'verify' subcommand", 2018-06-27) and the test it added it's intentional that "commit-graph verify" doesn't error out when the file doesn't exist. But let's not be overly promiscuous in what we accept. If we can't read the file for other reasons, e.g. permission errors, bad file descriptor etc. we'd like to report an error to the user. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph: don't pass filename to load_commit_graph_one_fd_st()Libravatar Ævar Arnfjörð Bjarmason3-8/+6
An earlier change implemented load_commit_graph_one_fd_st() in a way that was bug-compatible with earlier code in terms of the "graph file %s is too small" error message printing out the path to the commit-graph (".git/objects/info/commit-graph"). But change that, because: * A function that takes an already-open file descriptor also needing the filename isn't very intuitive. * The vast majority of errors we might emit when loading the graph come from parse_commit_graph(), which doesn't report the filename. Let's not do that either in this case for consistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph: don't early exit(1) on e.g. "git status"Libravatar Ævar Arnfjörð Bjarmason4-18/+51
Make the commit-graph loading code work as a library that returns an error code instead of calling exit(1) when the commit-graph is corrupt. This means that e.g. "status" will now report commit-graph corruption as an "error: [...]" at the top of its output, but then proceed to work normally. This required splitting up the load_commit_graph_one() function so that the code that deals with open()-ing and stat()-ing the graph can now be called independently as open_commit_graph(). This is needed because "commit-graph verify" where the graph doesn't exist isn't an error. See the third paragraph in 283e68c72f ("commit-graph: add 'verify' subcommand", 2018-06-27). There's a bug in that logic where we conflate the intended ENOENT with other errno values (e.g. EACCES), but this change doesn't address that. That'll be addressed in a follow-up change. I'm then splitting most of the logic out of load_commit_graph_one() into load_commit_graph_one_fd_st(), which allows for providing an existing file descriptor and stat information to the loading code. This isn't strictly needed, but it would be redundant and confusing to open() and stat() the file twice for some of the codepaths, this allows for calling open_commit_graph() followed by load_commit_graph_one_fd_st(). The "graph_file" still needs to be passed to that function for the the "graph file %s is too small" error message. This leaves load_commit_graph_one() unused by everything except the internal prepare_commit_graph_one() function, so let's mark it as "static". If someone needs it in the future we can remove the "static" attribute. I could also rewrite its sole remaining user ("prepare_commit_graph_one()") to use load_commit_graph_one_fd_st() instead, but let's leave it at this. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01commit-graph: fix segfault on e.g. "git status"Libravatar Ævar Arnfjörð Bjarmason2-10/+36
When core.commitGraph=true is set, various common commands now consult the commit graph. Because the commit-graph code is very trusting of its input data, it's possibly to construct a graph that'll cause an immediate segfault on e.g. "status" (and e.g. "log", "blame", ...). In some other cases where git immediately exits with a cryptic error about the graph being broken. The root cause of this is that while the "commit-graph verify" sub-command exhaustively verifies the graph, other users of the graph simply trust the graph, and will e.g. deference data found at certain offsets as pointers, causing segfaults. This change does the bare minimum to ensure that we don't segfault in the common fill_commit_in_graph() codepath called by e.g. setup_revisions(), to do this instrument the "commit-graph verify" tests to always check if "status" would subsequently segfault. This fixes the following tests which would previously segfault: not ok 50 - detect low chunk count not ok 51 - detect missing OID fanout chunk not ok 52 - detect missing OID lookup chunk not ok 53 - detect missing commit data chunk Those happened because with the commit-graph enabled setup_revisions() would eventually call fill_commit_in_graph(), where e.g. g->chunk_commit_data is used early as an offset (and will be 0x0). With this change we get far enough to detect that the graph is broken, and show an error instead. E.g.: $ git status; echo $? error: commit-graph is missing the Commit Data chunk 1 That also sucks, we should *warn* and not hard-fail "status" just because the commit-graph is corrupt, but fixing is left to a follow-up change. A side-effect of changing the reporting from graph_report() to error() is that we now have an "error: " prefix for these even for "commit-graph verify". Pseudo-diff before/after: $ git commit-graph verify -commit-graph is missing the Commit Data chunk +error: commit-graph is missing the Commit Data chunk Changing that is OK. Various errors it emits now early on are prefixed with "error: ", moving these over and changing the output doesn't break anything. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22commit-graph tests: test a graph that's too smallLibravatar Ævar Arnfjörð Bjarmason1-0/+6
Use the recently split-up components of the corrupt_graph_and_verify() function to assert that we error on graphs that are too small. The error was added in 2a2e32bdc5 ("commit-graph: implement git commit-graph read", 2018-04-10), but there was no test for it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22commit-graph tests: split up corrupt_graph_and_verify()Libravatar Ævar Arnfjörð Bjarmason1-6/+16
Split up the corrupt_graph_and_verify() function added in d9b9f8a6fd ("commit-graph: verify catches corrupt signature", 2018-06-27) into its logical components of setting up the test itself, doing the corruption in a particular way with "dd", and then finally testing that stderr is what we expect. This allows for re-using everything except the now slimmer corrupt_graph_and_verify() to corrupt the graph in a way that doesn't involve inserting a given byte sequence at a given position, e.g. truncating it entirely to a custom value. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19Git 2.21-rc2Libravatar Junio C Hamano1-1/+1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19Merge branch 'js/test-tool-gen-nuls'Libravatar Junio C Hamano5-7/+25
* js/test-tool-gen-nuls: tests: teach the test-tool to generate NUL bytes and use it
2019-02-19Merge branch 'mk/t5562-no-input-to-too-large-an-input-test'Libravatar Junio C Hamano1-2/+2
* mk/t5562-no-input-to-too-large-an-input-test: t5562: do not depend on /dev/zero Revert "t5562: replace /dev/zero with a pipe from generate_zero_bytes"
2019-02-19Merge branch 'mk/t5562-do-not-reuse-output-files'Libravatar Junio C Hamano1-4/+4
* mk/t5562-do-not-reuse-output-files: t5562: do not reuse output files
2019-02-19t5562: do not reuse output filesLibravatar Max Kirillov1-4/+4
Some expected failures of git-http-backend leaves running its children (receive-pack or upload-pack) which still hold opened descriptors to act.err and with some probability they live long enough to write there their failure messages after next test has already truncated the files. This causes occasional failures of the test script. Avoid the issue by using separated output and error file for each test, apprending the test number to their name. Reported-by: Carlo Arenas <carenas@gmail.com> Helped-by: Carlo Arenas <carenas@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19tests: teach the test-tool to generate NUL bytes and use itLibravatar Johannes Schindelin5-7/+25
In cc95bc2025 (t5562: replace /dev/zero with a pipe from generate_zero_bytes, 2019-02-09), we replaced usage of /dev/zero (which is not available on NonStop, apparently) by a Perl script snippet to generate NUL bytes. Sadly, it does not seem to work on NonStop, as t5562 reportedly hangs. Worse, this also hangs in the Ubuntu 16.04 agents of the CI builds on Azure Pipelines: for some reason, the Perl script snippet that is run via `generate_zero_bytes` in t5562's 'CONTENT_LENGTH overflow ssite_t' test case tries to write out an infinite amount of NUL bytes unless a broken pipe is encountered, that snippet never encounters the broken pipe, and keeps going until the build times out. Oddly enough, this does not reproduce on the Windows and macOS agents, nor in a local Ubuntu 18.04. This developer tried for a day to figure out the exact circumstances under which this hang happens, to no avail, the details remain a mystery. In the end, though, what counts is that this here change incidentally fixes that hang (maybe also on NonStop?). Even more positively, it gets rid of yet another unnecessary Perl invocation. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19t5562: do not depend on /dev/zeroLibravatar Max Kirillov1-1/+1
It was reported [1] that NonStop platform does not have /dev/zero. The test uses /dev/zero as a dummy input. Passing case (http-backed failed because of too big input size) should not be reading anything from it. If http-backend would erroneously try to read any data returning EOF probably would be even safer than providing some meaningless data. Replace /dev/zero with /dev/null to avoid issues with platforms which do not have /dev/zero. [1] https://public-inbox.org/git/20190209185930.5256-4-randall.s.becker@rogers.com/ Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19Revert "t5562: replace /dev/zero with a pipe from generate_zero_bytes"Libravatar Junio C Hamano1-2/+2
Revert cc95bc20 ("t5562: replace /dev/zero with a pipe from generate_zero_bytes", 2019-02-09), as not feeding anything to the command is a better way to test it.
2019-02-15mingw: safe-guard a bit more against getenv() problemsLibravatar Johannes Schindelin1-1/+1
Running up to v2.21.0, we fixed two bugs that were made prominent by the Windows-specific change to retain copies of only the 30 latest getenv() calls' returned strings, invalidating any copies of previous getenv() calls' return values. While this really shines a light onto bugs of the form where we hold onto getenv()'s return values without copying them, it is also a real problem for users. And even if Jeff King's patches merged via 773e408881 (Merge branch 'jk/save-getenv-result', 2019-01-29) provide further work on that front, we are far from done. Just one example: on Windows, we unset environment variables when spawning new processes, which potentially invalidates strings that were previously obtained via getenv(), and therefore we have to duplicate environment values that are somehow involved in spawning new processes (e.g. GIT_MAN_VIEWER in show_man_page()). We do not have a chance to investigate, let address, all of those issues in time for v2.21.0, so let's at least help Windows users by increasing the number of getenv() calls' return values that are kept valid. The number 64 was determined by looking at the average number of getenv() calls per process in the entire test suite run on Windows (which is around 40) and then adding a bit for good measure. And it is a power of two (which would have hit yesterday's theme perfectly). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14Merge branch 'ea/rebase-compat-doc-fix'Libravatar Junio C Hamano1-1/+0
* ea/rebase-compat-doc-fix: docs/git-rebase: remove redundant entry in incompatible options list
2019-02-14Merge branch 'jc/no-grepping-for-strerror-in-tests'Libravatar Junio C Hamano1-1/+1
* jc/no-grepping-for-strerror-in-tests: t1404: do not rely on the exact phrasing of strerror()
2019-02-14Merge branch 'jt/fetch-v2-sideband'Libravatar Junio C Hamano1-0/+1
"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache
2019-02-14Merge branch 'en/rebase-merge-on-sequencer'Libravatar Junio C Hamano1-2/+0
"git rebase --merge" as been reimplemented by reusing the internal machinery used for "git rebase -i". * en/rebase-merge-on-sequencer: git-rebase.txt: update to reflect merge now implemented on sequencer
2019-02-14git-rebase.txt: update to reflect merge now implemented on sequencerLibravatar Elijah Newren1-2/+0
Since commit 8fe9c3f21dff (Merge branch 'en/rebase-merge-on-sequencer', 2019-02-06), --merge now uses the interactive backend (and matches its behavior) so there is no separate merge backend anymore. Fix an oversight in the docs that should have been updated with the previous change. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through ApacheLibravatar Todd Zullinger1-0/+1
07c3c2aa16 ("tests: define GIT_TEST_SIDEBAND_ALL", 2019-01-16) added GIT_TEST_SIDEBAND_ALL to the apache.conf PassEnv list. Avoid warnings from Apache when the variable is unset, as we do for GIT_VALGRIND* and GIT_TRACE, from f628825481 ("t/lib-httpd: handle running under --valgrind", 2012-07-24) and 89c57ab3f0 ("t: pass GIT_TRACE through Apache", 2015-03-13), respectively. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14t1404: do not rely on the exact phrasing of strerror()Libravatar Junio C Hamano1-1/+1
Not even in C locale, it is wrong to expect that the exact phrasing "File exists" is used to show EEXIST. Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14docs/git-rebase: remove redundant entry in incompatible options listLibravatar Emilio Cobos Álvarez1-1/+0
The --autosquash option is implied by the earlier --[no-]autosquash entry in the list. Signed-off-by: Emilio Cobos Álvarez <emilio@crisal.io> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13Git 2.21-rc1Libravatar Junio C Hamano1-0/+10
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13Merge branch 'ab/rebase-test-fix'Libravatar Junio C Hamano1-3/+6
* ab/rebase-test-fix: rebase: fix regression in rebase.useBuiltin=false test mode