summaryrefslogtreecommitdiff
path: root/remote-curl.c
AgeCommit message (Collapse)AuthorFilesLines
2021-11-25run-command API users: use strvec_pushv(), not argv assignmentLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Migrate those run-command API users that assign directly to the "argv" member to use a strvec_pushv() of "args" instead. In these cases it did not make sense to further refactor these callers, e.g. daemon.c could be made to construct the arguments closer to handle(), but that would require moving the construction from its cmd_main() and pass "argv" through two intermediate functions. It would be possible for a change like this to introduce a regression if we were doing: cp.argv = argv; argv[1] = "foo"; And changed the code, as is being done here, to: strvec_pushv(&cp.args, argv); argv[1] = "foo"; But as viewing this change with the "-W" flag reveals none of these functions modify variable that's being pushed afterwards in a way that would introduce such a logic error. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-25Merge branch 'ab/pkt-line-cleanup'Libravatar Junio C Hamano1-1/+1
Code clean-up. * ab/pkt-line-cleanup: pkt-line.[ch]: remove unused packet_read_line_buf() pkt-line.[ch]: remove unused packet_buf_write_len()
2021-10-15pkt-line.[ch]: remove unused packet_read_line_buf()Libravatar Ævar Arnfjörð Bjarmason1-1/+1
This function was added in 4981fe750b1 (pkt-line: share buffer/descriptor reading implementation, 2013-02-23), but in 01f9ec64c8a (Use packet_reader instead of packet_read_line, 2018-12-29) the code that was using it was removed. Since it's being removed we can in turn remove the "src" and "src_len" arguments to packet_read(), all the remaining users just passed a NULL/NULL pair to it. That function is only a thin wrapper for packet_read_with_status() which still needs those arguments, but for the thin packet_read() convenience wrapper we can do away with it for now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-11Merge branch 'ab/http-pinned-public-key-mismatch'Libravatar Junio C Hamano1-0/+4
HTTPS error handling updates. * ab/http-pinned-public-key-mismatch: http: check CURLE_SSL_PINNEDPUBKEYNOTMATCH when emitting errors
2021-09-28string-list.[ch]: remove string_list_init() compatibility functionLibravatar Ævar Arnfjörð Bjarmason1-2/+2
Remove this function left over to accommodate in-flight changes, see 770fedaf9fb (string-list.[ch]: add a string_list_init_{nodup,dup}(), 2021-07-01) for the recent change to add "string_list_init_{nodup,dup}()" initializers. There was only one user of the API left in remote-curl.c. I don't know why I didn't include this change to remote-curl.c in bc40dfb10a0 (string-list.h users: change to use *_{nodup,dup}(), 2021-07-01), perhaps I just missed it. In any case, let's change that one user to use the new API, as of writing this there are no in-flight changes that use, so this seems like a good time to drop this before we get any new users of this compatibility API. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27http: check CURLE_SSL_PINNEDPUBKEYNOTMATCH when emitting errorsLibravatar Ævar Arnfjörð Bjarmason1-0/+4
Change the error shown when a http.pinnedPubKey doesn't match to point the http.pinnedPubKey variable added in aeff8a61216 (http: implement public key pinning, 2016-02-15), e.g.: git -c http.pinnedPubKey=sha256/someNonMatchingKey ls-remote https://github.com/git/git.git fatal: unable to access 'https://github.com/git/git.git/' with http.pinnedPubkey configuration: SSL: public key does not match pinned public key! Before this we'd emit the exact same thing without the " with http.pinnedPubkey configuration". The advantage of doing this is that we're going to get a translated message (everything after the ":" is hardcoded in English in libcurl), and we've got a reference to the git-specific configuration variable that's causing the error. Unfortunately we can't test this easily, as there are no tests that require https:// in the test suite, and t/lib-httpd.sh doesn't know how to set up such tests. See [1] for the start of a discussion about what it would take to have divergent "t/lib-httpd/apache.conf" test setups. #leftoverbits 1. https://lore.kernel.org/git/YUonS1uoZlZEt+Yd@coredump.intra.peff.net/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-24Merge branch 'ab/http-drop-old-curl'Libravatar Junio C Hamano1-9/+2
Support for ancient versions of cURL library (pre 7.19.4) has been dropped. * ab/http-drop-old-curl: http: rename CURLOPT_FILE to CURLOPT_WRITEDATA http: drop support for curl < 7.19.3 and < 7.17.0 (again) http: drop support for curl < 7.19.4 http: drop support for curl < 7.16.0 http: drop support for curl < 7.11.1
2021-07-30http: rename CURLOPT_FILE to CURLOPT_WRITEDATALibravatar Ævar Arnfjörð Bjarmason1-2/+2
The CURLOPT_FILE name is an alias for CURLOPT_WRITEDATA, the CURLOPT_WRITEDATA name has been preferred since curl 7.9.7, released in May 2002[1]. 1. https://curl.se/libcurl/c/CURLOPT_WRITEDATA.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-30http: drop support for curl < 7.16.0Libravatar Jeff King1-4/+0
In the last commit we dropped support for curl < 7.11.1, let's continue that and drop support for versions older than 7.16.0. This allows us to get rid of some now-obsolete #ifdefs. Choosing 7.16.0 is a somewhat arbitrary cutoff: 1. It came out in October of 2006, almost 15 years ago. Besides being a nice round number, around 10 years is a common end-of-life support period, even for conservative distributions. 2. That version introduced the curl_multi interface, which gives us a lot of bang for the buck in removing #ifdefs RHEL 5 came with curl 7.15.5[1] (released in August 2006). RHEL 5's extended life cycle program ended on 2020-11-30[1]. RHEL 6 comes with curl 7.19.7 (released in November 2009), and RHEL 7 comes with 7.29.0 (released in February 2013). 1. http://lore.kernel.org/git/873e1f31-2a96-5b72-2f20-a5816cad1b51@jupiterrise.com Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-30http: drop support for curl < 7.11.1Libravatar Jeff King1-3/+0
Drop support for this ancient version of curl and simplify the code by allowing us get rid of some "#ifdef"'s. Git will not build with vanilla curl older than 7.11.1 due our use of CURLOPT_POSTFIELDSIZE in 37ee680d9b (http.postbuffer: allow full range of ssize_t values, 2017-04-11). This field was introduced in curl 7.11.1. We could solve these compilation problems with more #ifdefs, but it's not worth the trouble. Version 7.11.1 came out in March of 2004, over 17 years ago. Let's declare that too old and drop any existing ifdefs that go further back. One obvious benefit is that we'll have fewer conditional bits cluttering the code. This patch drops all #ifdefs that reference older versions (note that curl's preprocessor macros are in hex, so we're looking for 070b01, not 071101). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-28Merge branch 'dl/packet-read-response-end-fix'Libravatar Junio C Hamano1-1/+1
Error message update. * dl/packet-read-response-end-fix: pkt-line: replace "stateless separator" with "response end"
2021-07-09pkt-line: replace "stateless separator" with "response end"Libravatar Denton Liu1-1/+1
In 0181b600a6 (pkt-line: define PACKET_READ_RESPONSE_END, 2020-05-19), the Response End packet was defined for Git's network protocol. When the patch was sent, it included an oversight where the error messages referenced "stateless separator", the work-in-progress name, over "response end", the final name chosen. Correct these error messages by having them correctly reference a "response end" packet. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-12remote-curl: fix clone on sha256 reposLibravatar Eric Wong1-0/+2
The remote-https process needs to update it's own instance of `the_repository' when it sees an HTTP(S) remote is using sha256. Without this, parse_oid_hex() fails to handle sha256 OIDs when it's eventually called by parse_fetch(). Tested with: git clone https://yhbt.net/sha256test.git GIT_SMART_HTTP=0 git clone https://yhbt.net/sha256test.git (plain http:// also works) Cloning the URL via git:// required no changes Signed-off-by: Eric Wong <e@80x24.org> Acked-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-03push: parse and set flag for "--force-if-includes"Libravatar Srinidhi Kaushik1-1/+13
The previous commit added the necessary machinery to implement the "--force-if-includes" protection, when "--force-with-lease" is used without giving exact object the remote still ought to have. Surface the feature by adding a command line option and a configuration variable to enable it. - Add a flag: "TRANSPORT_PUSH_FORCE_IF_INCLUDES" to indicate that the new option was passed from the command line of via configuration settings; update command line and configuration parsers to set the new flag accordingly. - Introduce a new configuration option "push.useForceIfIncludes", which is equivalent to setting "--force-if-includes" in the command line. - Update "remote-curl" to recognize and pass this option to "send-pack" when enabled. - Update "advise" to catch the reject reason "REJECT_REF_NEEDS_UPDATE", set when the ref status is "REF_STATUS_REJECT_REMOTE_UPDATED" and (optionally) print a help message when the push fails. - The new option is a "no-op" in the following scenarios: * When used without "--force-with-lease". * When used with "--force-with-lease", and if the expected commit on the remote side is specified as an argument. Signed-off-by: Srinidhi Kaushik <shrinidhi.kaushik@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-03Merge branch 'jt/lazy-fetch'Libravatar Junio C Hamano1-6/+0
Updates to on-demand fetching code in lazily cloned repositories. * jt/lazy-fetch: fetch: no FETCH_HEAD display if --no-write-fetch-head fetch-pack: remove no_dependents code promisor-remote: lazy-fetch objects in subprocess fetch-pack: do not lazy-fetch during ref iteration fetch: only populate existing_refs if needed fetch: avoid reading submodule config until needed fetch: allow refspecs specified through stdin negotiator/noop: add noop fetch negotiator
2020-08-24(various): document from_promisor parameterLibravatar Jonathan Tan1-0/+3
88e2f9ed8e ("introduce fetch-object: fetch one promisor object", 2017-12-05) plumbed through the from_promisor parameter but did not document it everywhere it appeared. Add the documentation. (It also plumbed through the no_dependents parameter, but I have left that alone because it is being removed in a commit under review [1].) [1] https://lore.kernel.org/git/e8f16d69089a5011c355d5939c56fa53b7a1eb2d.1597184949.git.jonathantanmy@google.com/ Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-18fetch-pack: remove no_dependents codeLibravatar Jonathan Tan1-6/+0
Now that Git has switched to using a subprocess to lazy-fetch missing objects, remove the no_dependents code as it is no longer used. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10Merge branch 'jk/strvec'Libravatar Junio C Hamano1-51/+51
The argv_array API is useful for not just managing argv but any "vector" (NULL-terminated array) of strings, and has seen adoption to a certain degree. It has been renamed to "strvec" to reduce the barrier to adoption. * jk/strvec: strvec: rename struct fields strvec: drop argv_array compatibility layer strvec: update documention to avoid argv_array strvec: fix indentation in renamed calls strvec: convert remaining callers away from argv_array name strvec: convert more callers away from argv_array name strvec: convert builtin/ callers away from argv_array name quote: rename sq_dequote_to_argv_array to mention strvec strvec: rename files from argv-array to strvec argv-array: rename to strvec argv-array: use size_t for count and alloc
2020-07-30strvec: rename struct fieldsLibravatar Jeff King1-3/+3
The "argc" and "argv" names made sense when the struct was argv_array, but now they're just confusing. Let's rename them to "nr" (which we use for counts elsewhere) and "v" (which is rather terse, but reads well when combined with typical variable names like "args.v"). Note that we have to update all of the callers immediately. Playing tricks with the preprocessor is hard here, because we wouldn't want to rewrite unrelated tokens. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30Merge branch 'bc/push-cas-cquoted-refname' into masterLibravatar Junio C Hamano1-1/+5
Pushing a ref whose name contains non-ASCII character with the "--force-with-lease" option did not work over smart HTTP protocol, which has been corrected. * bc/push-cas-cquoted-refname: remote-curl: make --force-with-lease work with non-ASCII ref names
2020-07-28strvec: fix indentation in renamed callsLibravatar Jeff King1-4/+4
Code which split an argv_array call across multiple lines, like: argv_array_pushl(&args, "one argument", "another argument", "and more", NULL); was recently mechanically renamed to use strvec, which results in mis-matched indentation like: strvec_pushl(&args, "one argument", "another argument", "and more", NULL); Let's fix these up to align the arguments with the opening paren. I did this manually by sifting through the results of: git jump grep 'strvec_.*,$' and liberally applying my editor's auto-format. Most of the changes are of the form shown above, though I also normalized a few that had originally used a single-tab indentation (rather than our usual style of aligning with the open paren). I also rewrapped a couple of obvious cases (e.g., where previously too-long lines became short enough to fit on one), but I wasn't aggressive about it. In cases broken to three or more lines, the grouping of arguments is sometimes meaningful, and it wasn't worth my time or reviewer time to ponder each case individually. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: convert remaining callers away from argv_array nameLibravatar Jeff King1-43/+43
We eventually want to drop the argv_array name and just use strvec consistently. There's no particular reason we have to do it all at once, or care about interactions between converted and unconverted bits. Because of our preprocessor compat layer, the names are interchangeable to the compiler (so even a definition and declaration using different names is OK). This patch converts all of the remaining files, as the resulting diff is reasonably sized. The conversion was done purely mechanically with: git ls-files '*.c' '*.h' | xargs perl -i -pe ' s/ARGV_ARRAY/STRVEC/g; s/argv_array/strvec/g; ' We'll deal with any indentation/style fallouts separately. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: rename files from argv-array to strvecLibravatar Jeff King1-1/+1
This requires updating #include lines across the code-base, but that's all fairly mechanical, and was done with: git ls-files '*.c' '*.h' | xargs perl -i -pe 's/argv-array.h/strvec.h/' Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-20remote-curl: make --force-with-lease work with non-ASCII ref namesLibravatar brian m. carlson1-1/+5
When we invoke a remote transport helper and pass an option with an argument, we quote the argument as a C-style string if necessary. This is the case for the cas option, which implements the --force-with-lease command-line flag, when we're passing a non-ASCII refname. However, the remote curl helper isn't designed to parse such an argument, meaning that if we try to use --force-with-lease with an HTTP push and a non-ASCII refname, we get an error like this: error: cannot parse expected object name '0000000000000000000000000000000000000000"' Note the double quote, which get_oid has reminded us is not valid in an hex object ID. Even if we had been able to parse it, we would send the wrong data to the server: we'd send an escaped ref, which would not behave as the user wanted and might accidentally result in updating or deleting a ref we hadn't intended. Since we need to expect a quoted C-style string here, just check if the first argument is a double quote, and if so, unquote it. Note that if the refname contains a double quote, then we will have double-quoted it already, so there is no ambiguity. We test for this case only in the smart protocol, since the DAV-based protocol is not capable of handling this capability. We use UTF-8 because this is nicer in our tests and friendlier to Windows, but the code should work for all non-ASCII refs. While we're at it, since the name of the option is now well established and isn't going to change, let's inline it instead of using the #define constant. Reported-by: Frej Bjon <frej.bjon@nemit.fi> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-06Merge branch 'bc/sha-256-part-2'Libravatar Junio C Hamano1-4/+42
SHA-256 migration work continues. * bc/sha-256-part-2: (44 commits) remote-testgit: adapt for object-format bundle: detect hash algorithm when reading refs t5300: pass --object-format to git index-pack t5704: send object-format capability with SHA-256 t5703: use object-format serve option t5702: offer an object-format capability in the test t/helper: initialize the repository for test-sha1-array remote-curl: avoid truncating refs with ls-remote t1050: pass algorithm to index-pack when outside repo builtin/index-pack: add option to specify hash algorithm remote-curl: detect algorithm for dumb HTTP by size builtin/ls-remote: initialize repository based on fetch t5500: make hash independent serve: advertise object-format capability for protocol v2 connect: parse v2 refs with correct hash algorithm connect: pass full packet reader when parsing v2 refs Documentation/technical: document object-format for protocol v2 t1302: expect repo format version 1 for SHA-256 builtin/show-index: provide options to determine hash algo t5302: modernize test formatting ...
2020-06-19remote-curl: avoid truncating refs with ls-remoteLibravatar brian m. carlson1-1/+3
Normally, the remote-curl transport helper is aware of the hash algorithm we're using because we're in a repo with the appropriate hash algorithm set. However, when using git ls-remote outside of a repository, we won't have initialized the hash algorithm properly, so use hash_to_hex_algop to print the ref corresponding to the algorithm we've detected. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-19remote-curl: detect algorithm for dumb HTTP by sizeLibravatar brian m. carlson1-2/+21
When reading the info/refs file for a repository, we have no explicit way to detect which hash algorithm is in use because the file doesn't provide one. Detect the hash algorithm in use by the size of the first object ID. If we have an empty repository, we don't know what the hash algorithm is on the remote side, so default to whatever the local side has configured. Without doing this, we cannot clone an empty repository since we don't know its hash algorithm. Test this case appropriately, since we currently have no tests for cloning an empty repository with the dumb HTTP protocol. We anonymize the URL like elsewhere in the function in case the user has decided to include a secret in the URL. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-27remote-curl: implement object-format extensionsLibravatar brian m. carlson1-1/+18
Implement the object-format extensions that let us determine the hash algorithm in use when pushing, pulling, and fetching. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-24stateless-connect: send response end packetLibravatar Denton Liu1-0/+5
Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated before the transaction is complete, remote-curl will blindly forward the packets before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the transaction and continue reading, expecting more input to continue the transaction. This results in a deadlock between the two processes. This can be seen in the following command which does not terminate: $ git -c protocol.version=2 clone https://github.com/git/git.git --shallow-since=20151012 Cloning into 'git'... whereas the v1 version does terminate as expected: $ git -c protocol.version=1 clone https://github.com/git/git.git --shallow-since=20151012 Cloning into 'git'... fatal: the remote end hung up unexpectedly Instead of blindly forwarding packets, make remote-curl insert a response end packet after proxying the responses from the remote server when using stateless_connect(). On the RPC client side, ensure that each response ends as described. A separate control packet is chosen because we need to be able to differentiate between what the remote server sends and remote-curl's control packets. By ensuring in the remote-curl code that a server cannot send response end packets, we prevent a malicious server from being able to perform a denial of service attack in which they spoof a response end packet and cause the described deadlock to happen. Reported-by: Force Charlie <charlieio@outlook.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-24pkt-line: define PACKET_READ_RESPONSE_ENDLibravatar Denton Liu1-0/+2
In a future commit, we will use PACKET_READ_RESPONSE_END to separate messages proxied by remote-curl. To prepare for this, add the PACKET_READ_RESPONSE_END enum value. In switch statements that need a case added, die() or BUG() when a PACKET_READ_RESPONSE_END is unexpected. Otherwise, mirror how PACKET_READ_DELIM is implemented (especially in cases where packets are being forwarded). Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-24remote-curl: error on incomplete packetLibravatar Denton Liu1-3/+56
Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated with a partially written packet, remote-curl will blindly send the partially written packet before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the partial packet and continue reading, expecting more input. This results in a deadlock between the two processes. For a stateless connection, inspect packets before sending them and error out if a packet line packet is incomplete. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18remote-curl: remove label indentationLibravatar Denton Liu1-1/+1
In the codebase, labels are aligned to the leftmost column. Remove the space-indentation from `free_specs:` to conform to this. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18remote-curl: fix typoLibravatar Denton Liu1-1/+1
Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-30oid_array: rename source file from sha1-arrayLibravatar Jeff King1-1/+1
We renamed the actual data structure in 910650d2f8 (Rename sha1_array to oid_array, 2017-03-31), but the file is still called sha1-array. Besides being slightly confusing, it makes it more annoying to grep for leftover occurrences of "sha1" in various files, because the header is included in so many places. Let's complete the transition by renaming the source and header files (and fixing up a few comment references). I kept the "-" in the name, as that seems to be our style; cf. fc1395f4a4 (sha1_file.c: rename to use dash in file name, 2018-04-10). We also have oidmap.h and oidset.h without any punctuation, but those are "struct oidmap" and "struct oidset" in the code. We _could_ make this "oidarray" to match, but somehow it looks uglier to me because of the length of "array" (plus it would be a very invasive patch for little gain). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-09Merge branch 'rs/show-progress-in-dumb-http-fetch'Libravatar Junio C Hamano1-0/+1
"git fetch" over HTTP walker protocol did not show any progress output. We inherently do not know how much work remains, but still we can show something not to bore users. * rs/show-progress-in-dumb-http-fetch: remote-curl: show progress for fetches over dumb HTTP
2020-03-03remote-curl: show progress for fetches over dumb HTTPLibravatar René Scharfe1-0/+1
Fetching over dumb HTTP transport doesn't show any progress, even with the option --progress. If the connection is slow or there is a lot of data to get then this can take a long time while the user is left to wonder if git got stuck. We don't know the number of objects to fetch at the outset, but we can count the ones we got. Show an open-ended progress indicator based on that number if the user asked for it. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-31C: use skip_prefix() to avoid hardcoded string lengthLibravatar Junio C Hamano1-2/+3
We often skip an optional prefix in a string with a hardcoded constant, e.g. if (starts_with(string, "prefix")) string += 6; which is less error prone when written skip_prefix(string, "prefix", &string); Note that this changes a few error messages from "git reflog expire --expire=nonsense.timestamp", which used to complain by saying '--expire=nonsense.timestamp' is not a valid timestamp but with this change, we say 'nonsense.timestamp' is not a valid timestamp which is more technically correct (the string with --expire= as a prefix obviously cannot be a valid timestamp, but the error is about the part of the input without that prefix). Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-23Merge branch 'bc/smart-http-atomic-push'Libravatar Junio C Hamano1-1/+12
The atomic push over smart HTTP transport did not work, which has been corrected. * bc/smart-http-atomic-push: remote-curl: pass on atomic capability to remote side
2019-10-17remote-curl: pass on atomic capability to remote sideLibravatar brian m. carlson1-1/+12
When pushing more than one reference with the --atomic option, the server is supposed to perform a single atomic transaction to update the references, leaving them either all to succeed or all to fail. This works fine when pushing locally or over SSH, but when pushing over HTTP, we fail to pass the atomic capability to the remote side. In fact, we have not reported this capability to any remote helpers during the life of the feature. Now normally, things happen to work nevertheless, since we actually check for most types of failures, such as non-fast-forward updates, on the client side, and just abort the entire attempt. However, if the server side reports a problem, such as the inability to lock a ref, the transaction isn't atomic, because we haven't passed the appropriate capability over and the remote side has no way of knowing that we wanted atomic behavior. Fix this by passing the option from the transport code through to remote helpers, and from the HTTP remote helper down to send-pack. With this change, we can detect if the server side rejects the push and report back appropriately. Note the difference in the messages: the remote side reports "atomic transaction failed", while our own checking rejects pushes with the message "atomic push failed". Document the atomic option in the remote helper documentation, so other implementers can implement it if they like. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-15remote-curl: use argv_array in parse_push()Libravatar René Scharfe1-13/+9
Use argv_array to build an array of strings instead of open-coding it. This simplifies the code a bit. We also need to make the specs parameter of push(), push_dav() and push_git() const to match the argv member of the argv_array. That's fine, as all three only actually read from the specs array anyway. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-03i18n: fix typos found during l10n for git 2.22.0Libravatar Jiang Xin1-2/+2
Fix two typos introduced by the following commits: + 31fba9d3b4 (diff-parseopt: convert --[src|dst]-prefix, 2019-03-24) + ed8b4132c8 (remote-curl: mark all error messages for translation, 2019-03-05) Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25Merge branch 'bc/hash-transition-16'Libravatar Junio C Hamano1-5/+6
Conversion from unsigned char[20] to struct object_id continues. * bc/hash-transition-16: (35 commits) gitweb: make hash size independent Git.pm: make hash size independent read-cache: read data in a hash-independent way dir: make untracked cache extension hash size independent builtin/difftool: use parse_oid_hex refspec: make hash size independent archive: convert struct archiver_args to object_id builtin/get-tar-commit-id: make hash size independent get-tar-commit-id: parse comment record hash: add a function to lookup hash algorithm by length remote-curl: make hash size independent http: replace sha1_to_hex http: compute hash of downloaded objects using the_hash_algo http: replace hard-coded constant with the_hash_algo http-walker: replace sha1_to_hex http-push: remove remaining uses of sha1_to_hex http-backend: allow 64-character hex names http-push: convert to use the_hash_algo builtin/pull: make hash-size independent builtin/am: make hash size independent ...
2019-04-16Merge branch 'js/remote-curl-i18n'Libravatar Junio C Hamano1-25/+25
Error messages given from the http transport have been updated so that they can be localized. * js/remote-curl-i18n: remote-curl: mark all error messages for translation
2019-04-16Merge branch 'js/anonymize-remote-curl-diag'Libravatar Junio C Hamano1-6/+13
remote-http transport did not anonymize URLs reported in its error messages at places. * js/anonymize-remote-curl-diag: curl: anonymize URLs in error messages and warnings
2019-04-01remote-curl: make hash size independentLibravatar brian m. carlson1-5/+6
Change one hard-coded use of the constant 40 to a reference to the_hash_algo. In addition, switch a use of get_oid_hex to parse_oid_hex to avoid the need to use a constant. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07Merge branch 'jt/http-auth-proto-v2-fix'Libravatar Junio C Hamano1-205/+179
Unify RPC code for smart http in protocol v0/v1 and v2, which fixes a bug in the latter (lack of authentication retry) and generally improves the code base. * jt/http-auth-proto-v2-fix: remote-curl: use post_rpc() for protocol v2 also remote-curl: refactor reading into rpc_state's buf remote-curl: reduce scope of rpc_state.result remote-curl: reduce scope of rpc_state.stdin_preamble remote-curl: reduce scope of rpc_state.argv
2019-03-06remote-curl: mark all error messages for translationLibravatar Johannes Schindelin1-25/+25
Suggested by Jeff King. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-05curl: anonymize URLs in error messages and warningsLibravatar Johannes Schindelin1-6/+13
Just like 47abd85ba0 (fetch: Strip usernames from url's before storing them, 2009-04-17) and later 882d49ca5c (push: anonymize URL in status output, 2016-07-13), this change anonymizes URLs (read: strips them of user names and especially passwords) in user-facing error messages and warnings. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-03remote-curl: use post_rpc() for protocol v2 alsoLibravatar Jonathan Tan1-184/+149
When transmitting and receiving POSTs for protocol v0 and v1, remote-curl uses post_rpc() (and associated functions), but when doing the same for protocol v2, it uses a separate set of functions (proxy_rpc() and others). Besides duplication of code, this has caused at least one bug: the auth retry mechanism that was implemented in v0/v1 was not implemented in v2. To fix this issue and avoid it in the future, make remote-curl also use post_rpc() when handling protocol v2. Because line lengths are written to the HTTP request in protocol v2 (unlike in protocol v0/v1), this necessitates changes in post_rpc() and some of the functions it uses; perform these changes too. A test has been included to ensure that the code for both the unchunked and chunked variants of the HTTP request is exercised. Note: stateless_connect() has been updated to use the lower-level packet reading functions instead of struct packet_reader. The low-level control is necessary here because we cannot change the destination buffer of struct packet_reader while it is being used; struct packet_buffer has a peeking mechanism which relies on the destination buffer being present in between a peek and a read. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22trace2: create new combined trace facilityLibravatar Jeff Hostetler1-0/+7
Create a new unified tracing facility for git. The eventual intent is to replace the current trace_printf* and trace_performance* routines with a unified set of git_trace2* routines. In addition to the usual printf-style API, trace2 provides higer-level event verbs with fixed-fields allowing structured data to be written. This makes post-processing and analysis easier for external tools. Trace2 defines 3 output targets. These are set using the environment variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT". These may be set to "1" or to an absolute pathname (just like the current GIT_TRACE). * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command summary data. * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE. It extends the output with columns for the command process, thread, repo, absolute and relative elapsed times. It reports events for child process start/stop, thread start/stop, and per-thread function nesting. * GIT_TR2_EVENT is a new structured format. It writes event data as a series of JSON records. Calls to trace2 functions log to any of the 3 output targets enabled without the need to call different trace_printf* or trace_performance* routines. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>