summaryrefslogtreecommitdiff
path: root/builtin/receive-pack.c
AgeCommit message (Collapse)AuthorFilesLines
2022-02-18Merge branch 'cb/clear-quarantine-early-on-all-ref-update-errors'Libravatar Junio C Hamano1-0/+9
"receive-pack" checks if it will do any ref updates (various conditions could reject a push) before received objects are taken out of the temporary directory used for quarantine purposes, so that a push that is known-to-fail will not leave crufts that a future "gc" needs to clean up. * cb/clear-quarantine-early-on-all-ref-update-errors: receive-pack: purge temporary data if no command is ready to run
2022-02-09Merge branch 'ab/config-based-hooks-2'Libravatar Junio C Hamano1-2/+5
More "config-based hooks". * ab/config-based-hooks-2: run-command: remove old run_hook_{le,ve}() hook API receive-pack: convert push-to-checkout hook to hook.h read-cache: convert post-index-change to use hook.h commit: convert {pre-commit,prepare-commit-msg} hook to hook.h git-p4: use 'git hook' to run hooks send-email: use 'git hook run' for 'sendemail-validate' git hook run: add an --ignore-missing flag hooks: convert worktree 'post-checkout' hook to hook library hooks: convert non-worktree 'post-checkout' hook to hook library merge: convert post-merge to use hook.h am: convert applypatch-msg to use hook.h rebase: convert pre-rebase to use hook.h hook API: add a run_hooks_l() wrapper am: convert {pre,post}-applypatch to use hook.h gc: use hook library for pre-auto-gc hook hook API: add a run_hooks() wrapper hook: add 'run' subcommand
2022-02-05Merge branch 'jc/find-header'Libravatar Junio C Hamano1-23/+10
Code clean-up. * jc/find-header: receive-pack.c: consolidate find header logic
2022-02-01receive-pack: purge temporary data if no command is ready to runLibravatar Chen Bojun1-0/+9
When pushing a hidden ref, e.g.: $ git push origin HEAD:refs/hidden/foo "receive-pack" will reject our request with an error message like this: ! [remote rejected] HEAD -> refs/hidden/foo (deny updating a hidden ref) The remote side ("git-receive-pack") will not create the hidden ref as expected, but the pack file sent by "git-send-pack" is left inside the remote repository. I.e. the quarantine directory is not purged as it should be. Add a checkpoint before calling "tmp_objdir_migrate()" and after calling the "pre-receive" hook to purge that temporary data in the quarantine area when there is no command ready to run. The reason we do not add the checkpoint before the "pre-receive" hook, but after it, is that the "pre-receive" hook is called with a switch-off "skip_broken" flag, and all commands, even broken ones, should be fed by calling "feed_receive_hook()". Add a new test case in t5516 as well. Helped-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Helped-by: Teng Long <dyroneteng@gmail.com> Signed-off-by: Chen Bojun <bojun.cbj@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-07receive-pack: convert push-to-checkout hook to hook.hLibravatar Emily Shaffer1-2/+5
Move the push-to-checkout hook away from run-command.h to and over to the new hook.h library. This removes the last direct user of run_hook_le(), so we could remove that function now, but let's leave that to a follow-up cleanup commit. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-06receive-pack.c: consolidate find header logicLibravatar John Cai1-23/+10
There are two functions that have very similar logic of finding a header value. find_commit_header, and find_header. We can conslidate the logic by introducing a new function find_header_mem, which is equivalent to find_commit_header except it takes a len parameter that determines how many bytes will be read. find_commit_header and find_header can then both call find_header_mem. This reduces duplicate logic, as the logic for finding header values can now all live in one place. Signed-off-by: John Cai <johncai86@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-03Merge branch 'ns/tmp-objdir'Libravatar Junio C Hamano1-1/+1
New interface into the tmp-objdir API to help in-core use of the quarantine feature. * ns/tmp-objdir: tmp-objdir: disable ref updates when replacing the primary odb tmp-objdir: new API for creating temporary writable databases
2021-12-21Merge branch 'ak/protect-any-current-branch'Libravatar Junio C Hamano1-42/+50
"git fetch" without the "--update-head-ok" option ought to protect a checked out branch from getting updated, to prevent the working tree that checks it out to go out of sync. The code was written before the use of "git worktree" got widespread, and only checked the branch that was checked out in the current worktree, which has been updated. (originally called ak/fetch-not-overwrite-any-current-branch) * ak/protect-any-current-branch: branch: protect branches checked out in all worktrees receive-pack: protect current branch for bare repository worktree receive-pack: clean dead code from update_worktree() fetch: protect branches checked out in all worktrees worktree: simplify find_shared_symref() memory ownership model branch: lowercase error messages receive-pack: lowercase error messages fetch: lowercase error messages
2021-12-21Merge branch 'fs/ssh-signing-key-lifetime'Libravatar Junio C Hamano1-2/+4
Extend the signing of objects with SSH keys and learn to pay attention to the key validity time range when verifying. * fs/ssh-signing-key-lifetime: ssh signing: verify ssh-keygen in test prereq ssh signing: make fmt-merge-msg consider key lifetime ssh signing: make verify-tag consider key lifetime ssh signing: make git log verify key lifetime ssh signing: make verify-commit consider key lifetime ssh signing: add key lifetime test prereqs ssh signing: use sigc struct to pass payload t/fmt-merge-msg: make gpgssh tests more specific t/fmt-merge-msg: do not redirect stderr
2021-12-15Merge branch 'rj/receive-pack-avoid-sigpipe-during-status-reporting'Libravatar Junio C Hamano1-0/+2
When the "git push" command is killed while the receiving end is trying to report what happened to the ref update proposals, the latter used to die, due to SIGPIPE. The code now ignores SIGPIPE to increase our chances to run the post-receive hook after it happens. * rj/receive-pack-avoid-sigpipe-during-status-reporting: receive-pack: ignore SIGPIPE while reporting status to client
2021-12-09ssh signing: use sigc struct to pass payloadLibravatar Fabian Stelzer1-2/+4
To be able to extend the payload metadata with things like its creation timestamp or the creators ident we remove the payload parameters to check_signature() and use the already existing sigc->payload field instead, only adding the length field to the struct. This also allows us to get rid of the xmemdupz() calls in the verify functions. Since sigc is now used to input data as well as output the result move it to the front of the function list. - Add payload_length to struct signature_check - Populate sigc.payload/payload_len on all call sites - Remove payload parameters to check_signature() - Remove payload parameters to internal verify_* functions and use sigc instead - Remove xmemdupz() used for verbose output since payload is now already populated. Signed-off-by: Fabian Stelzer <fs@gigacodes.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-08tmp-objdir: new API for creating temporary writable databasesLibravatar Neeraj Singh1-1/+1
The tmp_objdir API provides the ability to create temporary object directories, but was designed with the goal of having subprocesses access these object stores, followed by the main process migrating objects from it to the main object store or just deleting it. The subprocesses would view it as their primary datastore and write to it. Here we add the tmp_objdir_replace_primary_odb function that replaces the current process's writable "main" object directory with the specified one. The previous main object directory is restored in either tmp_objdir_migrate or tmp_objdir_destroy. For the --remerge-diff usecase, add a new `will_destroy` flag in `struct object_database` to mark ephemeral object databases that do not require fsync durability. Add 'git prune' support for removing temporary object databases, and make sure that they have a name starting with tmp_ and containing an operation-specific name. Based-on-patch-by: Elijah Newren <newren@gmail.com> Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-01receive-pack: protect current branch for bare repository worktreeLibravatar Anders Kaseorg1-5/+3
A bare repository won’t have a working tree at "..", but it may still have separate working trees created with git worktree. We should protect the current branch of such working trees from being updated or deleted, according to receive.denyCurrentBranch. Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-01receive-pack: clean dead code from update_worktree()Libravatar Anders Kaseorg1-16/+7
update_worktree() can only be called with a non-NULL worktree parameter, because that’s the only case where we set do_update_worktree = 1. worktree->path is always initialized to non-NULL. Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-01worktree: simplify find_shared_symref() memory ownership modelLibravatar Anders Kaseorg1-22/+41
Storing the worktrees list in a static variable meant that find_shared_symref() had to rebuild the list on each call (which is inefficient when the call site is in a loop), and also that each call invalidated the pointer returned by the previous call (which is confusing). Instead, make it the caller’s responsibility to pass in the worktrees list and manage its lifetime. Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-01receive-pack: lowercase error messagesLibravatar Anders Kaseorg1-5/+5
Documentation/CodingGuidelines says “do not end error messages with a full stop” and “do not capitalize the first word”. Clean up existing messages, some of which we will be touching in later steps in the series, that deviate from these rules in this file, as a preparation for the main part of the topic. Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-25run-command API: remove "env" member, always use "env_array"Libravatar Ævar Arnfjörð Bjarmason1-5/+6
Remove the "env" member from "struct child_process" in favor of always using the "env_array". As with the preceding removal of "argv" in favor of "args" this gets rid of current and future oddities around memory management at the API boundary (see the amended API docs). For some of the conversions we can replace patterns like: child.env = env->v; With: strvec_pushv(&child.env_array, env->v); But for others we need to guard the strvec_pushv() with a NULL check, since we're not passing in the "v" member of a "struct strvec", e.g. in the case of tmp_objdir_env()'s return value. Ideally we'd rename the "env_array" member to simply "env" as a follow-up, since it and "args" are now inconsistent in not having an "_array" suffix, and seemingly without any good reason, unless we look at the history of how they came to be. But as we've currently got 122 in-tree hits for a "git grep env_array" let's leave that for now (and possibly forever). Doing that rename would be too disruptive. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-25run-command API users: use strvec_push(), not argv constructionLibravatar Ævar Arnfjörð Bjarmason1-19/+12
Change a pattern of hardcoding an "argv" array size, populating it and assigning to the "argv" member of "struct child_process" to instead use "strvec_push()" to add data to the "args" member. As noted in the preceding commit this moves us further towards being able to remove the "argv" member in a subsequent commit These callers could have used strvec_pushl(), but moving to strvec_push() makes the diff easier to read, and keeps the arguments aligned as before. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-25run-command API users: use strvec_pushl(), not argv constructionLibravatar Ævar Arnfjörð Bjarmason1-25/+13
Change a pattern of hardcoding an "argv" array size, populating it and assigning to the "argv" member of "struct child_process" to instead use "strvec_pushl()" to add data to the "args" member. This implements the same behavior as before in fewer lines of code, and moves us further towards being able to remove the "argv" member in a subsequent commit. Since we've entirely removed the "argv" variable(s) we can be sure that no potential logic errors of the type discussed in a preceding commit are being introduced here, i.e. ones where the local "argv" was being modified after the assignment to "struct child_process"'s "argv". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-10receive-pack: ignore SIGPIPE while reporting status to clientLibravatar Robin Jarry1-0/+2
Before running the post-receive hook, status info is reported back to the client. If a remote client exits before or during the status report, receive-pack is killed by SIGPIPE and post-receive is never executed. The post-receive hook is often used to send email notifications (see contrib/hooks/post-receive-email), update bug trackers, start automatic builds, etc. Not executing it after an interrupted yet "successful" push can lead to inconsistencies. Ignore SIGPIPE before reporting status to the client to increase the chances of post-receive running if pre-receive was successful. This does not guarantee 100% consistency but it should resist early disconnection by the client. Signed-off-by: Robin Jarry <robin@jarry.cc> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-25Merge branch 'fs/ssh-signing'Libravatar Junio C Hamano1-0/+4
Use ssh public crypto for object and push-cert signing. * fs/ssh-signing: ssh signing: test that gpg fails for unknown keys ssh signing: tests for logs, tags & push certs ssh signing: duplicate t7510 tests for commits ssh signing: verify signatures using ssh-keygen ssh signing: provide a textual signing_key_id ssh signing: retrieve a default key from ssh-agent ssh signing: add ssh key format and signing code ssh signing: add test prereqs ssh signing: preliminary refactoring and clean-up
2021-09-27hook.c users: use "hook_exists()" instead of "find_hook()"Libravatar Ævar Arnfjörð Bjarmason1-1/+1
Use the new hook_exists() function instead of find_hook() where the latter was called in boolean contexts. This make subsequent changes in a series where we further refactor the hook API clearer, as we won't conflate wanting to get the path of the hook with checking for its existence. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27hook.[ch]: move find_hook() from run-command.c to hook.cLibravatar Ævar Arnfjörð Bjarmason1-0/+1
Move the find_hook() function from run-command.c to a new hook.c library. This change establishes a stub library that's pretty pointless right now, but will see much wider use with Emily Shaffer's upcoming "configuration-based hooks" series. Eventually all the hook related code will live in hook.[ch]. Let's start that process by moving the simple find_hook() function over as-is. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-20Merge branch 'js/run-command-close-packs'Libravatar Junio C Hamano1-2/+1
The run-command API has been updated so that the callers can easily ask the file descriptors open for packfiles to be closed immediately before spawning commands that may trigger auto-gc. * js/run-command-close-packs: Close object store closer to spawning child processes run_auto_maintenance(): implicitly close the object store run-command: offer to close the object store before running run-command: prettify the `RUN_COMMAND_*` flags pull: release packs before fetching commit-graph: when closing the graph, also release the slab
2021-09-20Merge branch 'ab/serve-cleanup'Libravatar Junio C Hamano1-1/+2
Code clean-up around "git serve". * ab/serve-cleanup: upload-pack: document and rename --advertise-refs serve.[ch]: remove "serve_options", split up --advertise-refs code {upload,receive}-pack tests: add --advertise-refs tests serve.c: move version line to advertise_capabilities() serve: move transfer.advertiseSID check into session_id_advertise() serve.[ch]: don't pass "struct strvec *keys" to commands serve: use designated initializers transport: use designated initializers transport: rename "fetch" in transport_vtable to "fetch_refs" serve: mark has_capability() as static
2021-09-10ssh signing: verify signatures using ssh-keygenLibravatar Fabian Stelzer1-0/+4
To verify a ssh signature we first call ssh-keygen -Y find-principal to look up the signing principal by their public key from the allowedSignersFile. If the key is found then we do a verify. Otherwise we only validate the signature but can not verify the signers identity. Verification uses the gpg.ssh.allowedSignersFile (see ssh-keygen(1) "ALLOWED SIGNERS") which contains valid public keys and a principal (usually user@domain). Depending on the environment this file can be managed by the individual developer or for example generated by the central repository server from known ssh keys with push access. This file is usually stored outside the repository, but if the repository only allows signed commits/pushes, the user might choose to store it in the repository. To revoke a key put the public key without the principal prefix into gpg.ssh.revocationKeyring or generate a KRL (see ssh-keygen(1) "KEY REVOCATION LISTS"). The same considerations about who to trust for verification as with the allowedSignersFile apply. Using SSH CA Keys with these files is also possible. Add "cert-authority" as key option between the principal and the key to mark it as a CA and all keys signed by it as valid for this CA. See "CERTIFICATES" in ssh-keygen(1). Signed-off-by: Fabian Stelzer <fs@gigacodes.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09Close object store closer to spawning child processesLibravatar Johannes Schindelin1-2/+1
In many cases where we spawned child processes that _may_ trigger a repack, we explicitly closed the object store first (so that the `repack` process can delete the `.pack` files, which would otherwise not be possible on Windows since files cannot be deleted as long as they as still in use). Wherever possible, we now use the new `close_object_store` bit of the `run_command()` API, to delay closing the object store even further. This makes the code easier to maintain because it is now more obvious that we only release those file handles because of those child processes. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01connected: refactor iterator to return next object ID directlyLibravatar Patrick Steinhardt1-10/+7
The object ID iterator used by the connectivity checks returns the next object ID via an out-parameter and then uses a return code to indicate whether an item was found. This is a bit roundabout: instead of a separate error code, we can just return the next object ID directly and use `NULL` pointers as indicator that the iterator got no items left. Furthermore, this avoids a copy of the object ID. Refactor the iterator and all its implementations to return object IDs directly. This brings a tiny performance improvement when doing a mirror-fetch of a repository with about 2.3M refs: Benchmark #1: 328dc58b49919c43897240f2eabfa30be2ce32a4~: git-fetch Time (mean ± σ): 30.110 s ± 0.148 s [User: 27.161 s, System: 5.075 s] Range (min … max): 29.934 s … 30.406 s 10 runs Benchmark #2: 328dc58b49919c43897240f2eabfa30be2ce32a4: git-fetch Time (mean ± σ): 29.899 s ± 0.109 s [User: 26.916 s, System: 5.104 s] Range (min … max): 29.696 s … 29.996 s 10 runs Summary '328dc58b49919c43897240f2eabfa30be2ce32a4: git-fetch' ran 1.01 ± 0.01 times faster than '328dc58b49919c43897240f2eabfa30be2ce32a4~: git-fetch' While this 1% speedup could be labelled as statistically insignificant, the speedup is consistent on my machine. Furthermore, this is an end to end test, so it is expected that the improvement in the connectivity check itself is more significant. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-05upload-pack: document and rename --advertise-refsLibravatar Ævar Arnfjörð Bjarmason1-1/+2
The --advertise-refs documentation in git-upload-pack added in 9812f2136b3 (upload-pack.c: use parse-options API, 2016-05-31) hasn't been entirely true ever since v2 support was implemented in e52449b6722 (connect: request remote refs using v2, 2018-03-15). Under v2 we don't advertise the refs at all, but rather dump the capabilities header. This option has always been an obscure internal implementation detail, it wasn't even documented for git-receive-pack. Since it has exactly one user let's rename it to --http-backend-info-refs, which is more accurate and points the reader in the right direction. Let's also cross-link this from the protocol v1 and v2 documentation. I'm retaining a hidden --advertise-refs alias in case there's any external users of this, and making both options hidden to the bash completion (as with most other internal-only options). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-12*.c static functions: don't forward-declare __attribute__Libravatar Ævar Arnfjörð Bjarmason1-3/+2
9cf6d3357aa (Add git-index-pack utility, 2005-10-12) and 466dbc42f58 (receive-pack: Send internal errors over side-band #2, 2010-02-10) we added these static functions and forward-declared their __attribute__((printf)). I think this may have been to work around some compiler limitation at the time, but in any case we have a lot of code that uses the briefer way of declaring these that I'm using here, so if we had any such issues with compilers we'd have seen them already. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-27hash: provide per-algorithm null OIDsLibravatar brian m. carlson1-1/+1
Up until recently, object IDs did not have an algorithm member, only a hash. Consequently, it was possible to share one null (all-zeros) object ID among all hash algorithms. Now that we're going to be handling objects from multiple hash algorithms, it's important to make sure that all object IDs have a correct algorithm field. Introduce a per-algorithm null OID, and add it to struct hash_algo. Introduce a wrapper function as well, and use it everywhere we used to use the null_oid constant. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-19Merge branch 'rs/calloc-array'Libravatar Junio C Hamano1-2/+2
CALLOC_ARRAY() macro replaces many uses of xcalloc(). * rs/calloc-array: cocci: allow xcalloc(1, size) use CALLOC_ARRAY git-compat-util.h: drop trailing semicolon from macro definition
2021-03-19Merge branch 'jc/calloc-fix'Libravatar Junio C Hamano1-5/+3
Code clean-up. * jc/calloc-fix: xcalloc: use CALLOC_ARRAY() when applicable
2021-03-15xcalloc: use CALLOC_ARRAY() when applicableLibravatar Junio C Hamano1-5/+3
These are for codebase before Git 2.31 Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-13use CALLOC_ARRAYLibravatar René Scharfe1-2/+2
Add and apply a semantic patch for converting code that open-codes CALLOC_ARRAY to use it instead. It shortens the code and infers the element size automatically. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-01Merge branch 'jt/transfer-fsck-across-packs'Libravatar Junio C Hamano1-1/+1
The approach to "fsck" the incoming objects in "index-pack" is attractive for performance reasons (we have them already in core, inflated and ready to be inspected), but fundamentally cannot be applied fully when we receive more than one pack stream, as a tree object in one pack may refer to a blob object in another pack as ".gitmodules", when we want to inspect blobs that are used as ".gitmodules" file, for example. Teach "index-pack" to emit objects that must be inspected later and check them in the calling "fetch-pack" process. * jt/transfer-fsck-across-packs: fetch-pack: print and use dangling .gitmodules fetch-pack: with packfile URIs, use index-pack arg http-fetch: allow custom index-pack args http: allow custom index-pack args
2021-02-22fetch-pack: print and use dangling .gitmodulesLibravatar Jonathan Tan1-1/+1
Teach index-pack to print dangling .gitmodules links after its "keep" or "pack" line instead of declaring an error, and teach fetch-pack to check such lines printed. This allows the tree side of the .gitmodules link to be in one packfile and the blob side to be in another without failing the fsck check, because it is now fetch-pack which checks such objects after all packfiles have been downloaded and indexed (and not index-pack on an individual packfile, as it is before this commit). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-10gpg-interface: improve interface for parsing tagsLibravatar brian m. carlson1-2/+2
We have a function which parses a buffer with a signature at the end, parse_signature, and this function is used for signed tags. However, we'll need to store values for multiple algorithms, and we'll do this by using a header for the non-default algorithm. Adjust the parse_signature interface to store the parsed data in two strbufs and turn the existing function into parse_signed_buffer. The latter is still used in places where we know we always have a signed buffer, such as push certs. Adjust all the callers to deal with this new interface. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-12-08Merge branch 'js/trace2-session-id'Libravatar Junio C Hamano1-0/+15
The transport layer was taught to optionally exchange the session ID assigned by the trace2 subsystem during fetch/push transactions. * js/trace2-session-id: receive-pack: log received client session ID send-pack: advertise session ID in capabilities upload-pack, serve: log received client session ID fetch-pack: advertise session ID in capabilities transport: log received server session ID serve: advertise session ID in v2 capabilities receive-pack: advertise session ID in v0 capabilities upload-pack: advertise session ID in v0 capabilities trace2: add a public function for getting the SID docs: new transfer.advertiseSID option docs: new capability to advertise session IDs
2020-11-11receive-pack: log received client session IDLibravatar Josh Steadmon1-0/+7
When receive-pack receives a session-id capability from the client, log the received session ID via a trace2 data event. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-11receive-pack: advertise session ID in v0 capabilitiesLibravatar Josh Steadmon1-0/+8
When transfer.advertiseSID is true, advertise receive-pack's session ID via the new session-id capability. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-11receive-pack: use default version 0 for proc-receiveLibravatar Jiang Xin1-1/+6
In the verison negotiation phase between "receive-pack" and "proc-receive", "proc-receive" can send an empty flush-pkt to end the negotiation and use default version 0. Capabilities (such as "push-options") are not supported in version 0. Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-11receive-pack: gently write messages to proc-receiveLibravatar Jiang Xin1-24/+63
Johannes found a flaky hang in `t5411/test-0013-bad-protocol.sh` in the osx-clang job of the CI/PR builds, and ran into an issue when using the `--stress` option with the following error messages: fatal: unable to write flush packet: Broken pipe send-pack: unexpected disconnect while reading sideband packet fatal: the remote end hung up unexpectedly In this test case, the "proc-receive" hook sends an error message and dies earlier. While "receive-pack" on the other side of the pipe should forward the error message of the "proc-receive" hook to the client side, but it fails to do so. This is because "receive-pack" uses `packet_write_fmt()` and `packet_flush()` to write pkt-line message to "proc-receive" hook, and these functions die immediately when pipe is broken. Using "gently" forms for these functions will get more predicable output. Add more "--die-*" options to test helper to test different stages of the protocol between "receive-pack" and "proc-receive" hook. Reported-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25Merge branch 'jx/proc-receive-hook'Libravatar Junio C Hamano1-9/+463
"git receive-pack" that accepts requests by "git push" learned to outsource most of the ref updates to the new "proc-receive" hook. * jx/proc-receive-hook: doc: add documentation for the proc-receive hook transport: parse report options for tracking refs t5411: test updates of remote-tracking branches receive-pack: new config receive.procReceiveRefs doc: add document for capability report-status-v2 New capability "report-status-v2" for git-push receive-pack: feed report options to post-receive receive-pack: add new proc-receive hook t5411: add basic test cases for proc-receive hook transport: not report a non-head push as a branch
2020-08-27receive-pack: new config receive.procReceiveRefsLibravatar Jiang Xin1-7/+99
Add a new multi-valued config variable "receive.procReceiveRefs" for `receive-pack` command, like the follows: git config --system --add receive.procReceiveRefs refs/for git config --system --add receive.procReceiveRefs refs/drafts If the specific prefix strings given by the config variables match the reference names of the commands which are sent from git client to `receive-pack`, these commands will be executed by an external hook (named "proc-receive"), instead of the internal `execute_commands` function. For example, if it is set to "refs/for", pushing to a reference such as "refs/for/master" will not create or update reference "refs/for/master", but may create or update a pull request directly by running the hook "proc-receive". Optional modifiers can be provided in the beginning of the value to filter commands for specific actions: create (a), modify (m), delete (d). A `!` can be included in the modifiers to negate the reference prefix entry. E.g.: git config --system --add receive.procReceiveRefs ad:refs/heads git config --system --add receive.procReceiveRefs !:refs/heads Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-27New capability "report-status-v2" for git-pushLibravatar Jiang Xin1-2/+52
The new introduced "proc-receive" hook may handle a command for a pseudo-reference with a zero-old as its old-oid, while the hook may create or update a reference with different name, different new-oid, and different old-oid (the reference may exist already with a non-zero old-oid). Current "report-status" protocol cannot report the status for such reference rewrite. Add new capability "report-status-v2" and new report protocol which is not backward compatible for report of git-push. If a user pushes to a pseudo-reference "refs/for/master/topic", and "receive-pack" creates two new references "refs/changes/23/123/1" and "refs/changes/24/124/1", for client without the knowledge of "report-status-v2", "receive-pack" will only send "ok/ng" directives in the report, such as: ok ref/for/master/topic But for client which has the knowledge of "report-status-v2", "receive-pack" will use "option" directives to report more attributes for the reference given by the above "ok/ng" directive. ok refs/for/master/topic option refname refs/changes/23/123/1 option new-oid <new-oid> ok refs/for/master/topic option refname refs/changes/24/124/1 option new-oid <new-oid> The client will report two new created references to the end user. Suggested-by: Junio C Hamano <gitster@pobox.com> Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-27receive-pack: feed report options to post-receiveLibravatar Jiang Xin1-4/+26
When commands are fed to the "post-receive" hook, report options will be parsed and the real old-oid, new-oid, reference name will feed to the "post-receive" hook. Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-27receive-pack: add new proc-receive hookLibravatar Jiang Xin1-3/+293
Git calls an internal `execute_commands` function to handle commands sent from client to `git-receive-pack`. Regardless of what references the user pushes, git creates or updates the corresponding references if the user has write-permission. A contributor who has no write-permission, cannot push to the repository directly. So, the contributor has to write commits to an alternate location, and sends pull request by emails or by other ways. We call this workflow as a distributed workflow. It would be more convenient to work in a centralized workflow like what Gerrit provided for some cases. For example, a read-only user who cannot push to a branch directly can run the following `git push` command to push commits to a pseudo reference (has a prefix "refs/for/", not "refs/heads/") to create a code review. git push origin \ HEAD:refs/for/<branch-name>/<session> The `<branch-name>` in the above example can be as simple as "master", or a more complicated branch name like "foo/bar". The `<session>` in the above example command can be the local branch name of the client side, such as "my/topic". We cannot implement a centralized workflow elegantly by using "pre-receive" + "post-receive", because Git will call the internal function "execute_commands" to create references (even the special pseudo reference) between these two hooks. Even though we can delete the temporarily created pseudo reference via the "post-receive" hook, having a temporary reference is not safe for concurrent pushes. So, add a filter and a new handler to support this kind of workflow. The filter will check the prefix of the reference name, and if the command has a special reference name, the filter will turn a specific field (`run_proc_receive`) on for the command. Commands with this filed turned on will be executed by a new handler (a hook named "proc-receive") instead of the internal `execute_commands` function. We can use this "proc-receive" command to create pull requests or send emails for code review. Suggested by Junio, this "proc-receive" hook reads the commands, push-options (optional), and send result using a protocol in pkt-line format. In the following example, the letter "S" stands for "receive-pack" and letter "H" stands for the hook. # Version and features negotiation. S: PKT-LINE(version=1\0push-options atomic...) S: flush-pkt H: PKT-LINE(version=1\0push-options...) H: flush-pkt # Send commands from server to the hook. S: PKT-LINE(<old-oid> <new-oid> <ref>) S: ... ... S: flush-pkt # Send push-options only if the 'push-options' feature is enabled. S: PKT-LINE(push-option) S: ... ... S: flush-pkt # Receive result from the hook. # OK, run this command successfully. H: PKT-LINE(ok <ref>) # NO, I reject it. H: PKT-LINE(ng <ref> <reason>) # Fall through, let 'receive-pack' to execute it. H: PKT-LINE(ok <ref>) H: PKT-LINE(option fall-through) # OK, but has an alternate reference. The alternate reference name # and other status can be given in options H: PKT-LINE(ok <ref>) H: PKT-LINE(option refname <refname>) H: PKT-LINE(option old-oid <old-oid>) H: PKT-LINE(option new-oid <new-oid>) H: PKT-LINE(option forced-update) H: ... ... H: flush-pkt After receiving a command, the hook will execute the command, and may create/update different reference. For example, a command for a pseudo reference "refs/for/master/topic" may create/update different reference such as "refs/pull/123/head". The alternate reference name and other status are given in option lines. The list of commands returned from "proc-receive" will replace the relevant commands that are sent from user to "receive-pack", and "receive-pack" will continue to run the "execute_commands" function and other routines. Finally, the result of the execution of these commands will be reported to end user. The reporting function from "receive-pack" to "send-pack" will be extended in latter commit just like what the "proc-receive" hook reports to "receive-pack". Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30strvec: rename struct fieldsLibravatar Jeff King1-7/+7
The "argc" and "argv" names made sense when the struct was argv_array, but now they're just confusing. Let's rename them to "nr" (which we use for counts elsewhere) and "v" (which is rather terse, but reads well when combined with typical variable names like "args.v"). Note that we have to update all of the callers immediately. Playing tricks with the preprocessor is hard here, because we wouldn't want to rewrite unrelated tokens. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: fix indentation in renamed callsLibravatar Jeff King1-21/+21
Code which split an argv_array call across multiple lines, like: argv_array_pushl(&args, "one argument", "another argument", "and more", NULL); was recently mechanically renamed to use strvec, which results in mis-matched indentation like: strvec_pushl(&args, "one argument", "another argument", "and more", NULL); Let's fix these up to align the arguments with the opening paren. I did this manually by sifting through the results of: git jump grep 'strvec_.*,$' and liberally applying my editor's auto-format. Most of the changes are of the form shown above, though I also normalized a few that had originally used a single-tab indentation (rather than our usual style of aligning with the open paren). I also rewrapped a couple of obvious cases (e.g., where previously too-long lines became short enough to fit on one), but I wasn't aggressive about it. In cases broken to three or more lines, the grouping of arguments is sometimes meaningful, and it wasn't worth my time or reviewer time to ponder each case individually. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>