summaryrefslogtreecommitdiff
path: root/Documentation/technical/pack-protocol.txt
AgeCommit message (Collapse)AuthorFilesLines
2016-10-10Merge branch 'nd/shallow-deepen'Libravatar Junio C Hamano1-1/+3
The existing "git fetch --depth=<n>" option was hard to use correctly when making the history of an existing shallow clone deeper. A new option, "--deepen=<n>", has been added to make this easier to use. "git clone" also learned "--shallow-since=<date>" and "--shallow-exclude=<tag>" options to make it easier to specify "I am interested only in the recent N months worth of history" and "Give me only the history since that version". * nd/shallow-deepen: (27 commits) fetch, upload-pack: --deepen=N extends shallow boundary by N commits upload-pack: add get_reachable_list() upload-pack: split check_unreachable() in two, prep for get_reachable_list() t5500, t5539: tests for shallow depth excluding a ref clone: define shallow clone boundary with --shallow-exclude fetch: define shallow boundary with --shallow-exclude upload-pack: support define shallow boundary by excluding revisions refs: add expand_ref() t5500, t5539: tests for shallow depth since a specific date clone: define shallow clone boundary based on time with --shallow-since fetch: define shallow boundary with --shallow-since upload-pack: add deepen-since to cut shallow repos based on time shallow.c: implement a generic shallow boundary finder based on rev-list fetch-pack: use a separate flag for fetch in deepening mode fetch-pack.c: mark strings for translating fetch-pack: use a common function for verbose printing fetch-pack: use skip_prefix() instead of starts_with() upload-pack: move rev-list code out of check_non_tip() upload-pack: make check_non_tip() clean things up on error upload-pack: tighten number parsing at "deepen" lines ...
2016-08-04Merge branch 'sb/pack-protocol-doc-nak'Libravatar Junio C Hamano1-1/+1
A doc update. * sb/pack-protocol-doc-nak: Documentation: pack-protocol correct NAK response
2016-07-22Documentation: pack-protocol correct NAK responseLibravatar Stefan Beller1-1/+1
In the transport protocol we use NAK to signal the non existence of a common base, so fix the documentation. This helps readers of the document, as they don't have to wonder about the difference between NAK and NACK. As NACK is used in git archive and upload-archive, this is easy to get wrong. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-14receive-pack: implement advertising and receiving push optionsLibravatar Stefan Beller1-4/+6
The pre/post receive hook may be interested in more information from the user. This information can be transmitted when both client and server support the "push-options" capability, which when used is a phase directly after update commands ended by a flush pkt. Similar to the atomic option, the server capability can be disabled via the `receive.advertisePushOptions` config variable. While documenting this, fix a nit in the `receive.advertiseAtomic` wording. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13upload-pack: support define shallow boundary by excluding revisionsLibravatar Nguyễn Thái Ngọc Duy1-1/+2
This should allow the user to say "create a shallow clone of this branch after version <some-tag>". Short refs are accepted and expanded at the server side with expand_ref() because we cannot expand (unknown) refs from the client side. Like deepen-since, deepen-not cannot be used with deepen. But deepen-not can be mixed with deepen-since. The result is exactly how you do the command "git rev-list --since=... --not ref". Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13upload-pack: add deepen-since to cut shallow repos based on timeLibravatar Nguyễn Thái Ngọc Duy1-1/+2
This should allow the user to say "create a shallow clone containing the work from last year" (once the client side is fixed up, of course). In theory deepen-since and deepen (aka --depth) can be used together to draw the shallow boundary (whether it's intersection or union is up to discussion, but if rev-list is used, it's likely intersection). However, because deepen goes with a custom commit walker, we can't mix the two yet. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06typofix: assorted typofixes in comments, documentation and messagesLibravatar Li Peng1-1/+1
Many instances of duplicate words (e.g. "the the path") and a few typoes are fixed, originally in multiple patches. wildmatch: fix duplicate words of "the" t: fix duplicate words of "output" transport-helper: fix duplicate words of "read" Git.pm: fix duplicate words of "return" path: fix duplicate words of "look" pack-protocol.txt: fix duplicate words of "the" precompose-utf8: fix typo of "sequences" split-index: fix typo worktree.c: fix typo remote-ext: fix typo utf8: fix duplicate words of "the" git-cvsserver: fix duplicate words Signed-off-by: Li Peng <lip@dtdream.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03pack-protocol: clarify LF-handling in PKT-LINE()Libravatar Jeff King1-20/+26
The spec is very inconsistent about which PKT-LINE() parts of the grammar include a LF. On top of that, the code is not consistent, either (e.g., send-pack does not put newlines into the ref-update commands it sends). Let's make explicit the long-standing expectation that we generally expect pkt-lines to end in a newline, but that receivers should be lenient. This makes the spec consistent, and matches what git already does (though it does not always fulfill the SHOULD). We do make an exception for the push-cert, where the receiving code is currently a bit pickier. This is a reasonable way to be, as the data needs to be byte-for-byte compatible with what was signed. We _could_ make up some rules about signing a canonicalized version including newlines, but that would require a code change, and is out of scope for this patch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-25Merge branch 'sb/pack-protocol-mention-smart-http' into maintLibravatar Junio C Hamano1-3/+3
Doc updates. * sb/pack-protocol-mention-smart-http: Documentation/technical/pack-protocol: mention http as possible protocol
2015-06-16Merge branch 'sb/pack-protocol-mention-smart-http'Libravatar Junio C Hamano1-3/+3
Doc updates. * sb/pack-protocol-mention-smart-http: Documentation/technical/pack-protocol: mention http as possible protocol
2015-06-02Documentation/technical/pack-protocol: mention http as possible protocolLibravatar Stefan Beller1-3/+3
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-22doc: fix inconsistent spelling of "packfile"Libravatar Patrick Steinhardt1-2/+2
Fix remaining instances where "pack-file" is used instead of "packfile". Some places remain where we still use "pack-file", This is the case when we explicitly refer to a file with a ".pack" extension as opposed to a data source providing a pack data stream. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-17pack-protocol.txt: fix insconsistent spelling of "packfile"Libravatar Patrick Steinhardt1-3/+3
Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-08Merge branch 'jc/push-cert'Libravatar Junio C Hamano1-3/+46
Allow "git push" request to be signed, so that it can be verified and audited, using the GPG signature of the person who pushed, that the tips of branches at a public repository really point the commits the pusher wanted to, without having to "trust" the server. * jc/push-cert: (24 commits) receive-pack::hmac_sha1(): copy the entire SHA-1 hash out signed push: allow stale nonce in stateless mode signed push: teach smart-HTTP to pass "git push --signed" around signed push: fortify against replay attacks signed push: add "pushee" header to push certificate signed push: remove duplicated protocol info send-pack: send feature request on push-cert packet receive-pack: GPG-validate push certificates push: the beginning of "git push --signed" pack-protocol doc: typofix for PKT-LINE gpg-interface: move parse_signature() to where it should be gpg-interface: move parse_gpg_output() to where it should be send-pack: clarify that cmds_sent is a boolean send-pack: refactor inspecting and resetting status and sending commands send-pack: rename "new_refs" to "need_pack_data" receive-pack: factor out capability string generation send-pack: factor out capability string generation send-pack: always send capabilities send-pack: refactor decision to send update per ref send-pack: move REF_STATUS_REJECT_NODELETE logic a bit higher ...
2014-09-17signed push: fortify against replay attacksLibravatar Junio C Hamano1-0/+6
In order to prevent a valid push certificate for pushing into an repository from getting replayed in a different push operation, send a nonce string from the receive-pack process and have the signer include it in the push certificate. The receiving end uses an HMAC hash of the path to the repository it serves and the current time stamp, hashed with a secret seed (the secret seed does not have to be per-repository but can be defined in /etc/gitconfig) to generate the nonce, in order to ensure that a random third party cannot forge a nonce that looks like it originated from it. The original nonce is exported as GIT_PUSH_CERT_NONCE for the hooks to examine and match against the value on the "nonce" header in the certificate to notice a replay, but returned "nonce" header in the push certificate is examined by receive-pack and the result is exported as GIT_PUSH_CERT_NONCE_STATUS, whose value would be "OK" if the nonce recorded in the certificate matches what we expect, so that the hooks can more easily check. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-15signed push: add "pushee" header to push certificateLibravatar Junio C Hamano1-0/+6
Record the URL of the intended recipient for a push (after anonymizing it if it has authentication material) on a new "pushee URL" header. Because the networking configuration (SSH-tunnels, proxies, etc.) on the pushing user's side varies, the receiving repository may not know the single canonical URL all the pushing users would refer it as (besides, many sites allow pushing over ssh://host/path and https://host/path protocols to the same repository but with different local part of the path). So this value may not be reliably used for replay-attack prevention purposes, but this will still serve as a human readable hint to identify the repository the certificate refers to. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-15signed push: remove duplicated protocol infoLibravatar Junio C Hamano1-1/+32
With the interim protocol, we used to send the update commands even though we already send a signed copy of the same information when push certificate is in use. Update the send-pack/receive-pack pair not to do so. The notable thing on the receive-pack side is that it makes sure that there is no command sent over the traditional protocol packet outside the push certificate. Otherwise a pusher can claim to be pushing one set of ref updates in the signed certificate while issuing commands to update unrelated refs, and such an update will evade later audits. Finally, start documenting the protocol. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-15pack-protocol doc: typofix for PKT-LINELibravatar Junio C Hamano1-2/+2
Everywhere else we use PKT-LINE to denote the pkt-line formatted data, but "shallow/deepen" messages are described with PKT_LINE(). Fix them. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-28Document LF appearing in shallow command during send-pack/receive-packLibravatar Shawn Pearce1-1/+1
The implementation sends an LF, but the protocol documentation was missing this detail. Signed-off-by: Shawn Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-31Documentation: fix misuses of "nor"Libravatar Justin Lebar1-4/+4
Signed-off-by: Justin Lebar <jlebar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-02-10pack-protocol.txt: clarify 'obj-id' in the last ACK after 'done'Libravatar Nguyễn Thái Ngọc Duy1-1/+2
It's introduced in 1bd8c8f (git-upload-pack: Support the multi_ack protocol - 2005-10-28) but probably better documented in the commit message of 78affc4 (Add multi_ack_detailed capability to fetch-pack/upload-pack - 2009-10-30). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10receive/send-pack: support pushing from a shallow cloneLibravatar Nguyễn Thái Ngọc Duy1-1/+3
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10make the sender advertise shallow commits to the receiverLibravatar Nguyễn Thái Ngọc Duy1-0/+3
If either receive-pack or upload-pack is called on a shallow repository, shallow commits (*) will be sent after the ref advertisement (but before the packet flush), so that the receiver has the full "shape" of the sender's commit graph. This will be needed for the receiver to update its .git/shallow if necessary. This breaks the protocol for all clients trying to push to a shallow repo, or fetch from one. Which is basically the same end result as today's "is_repository_shallow() && die()" in receive-pack and upload-pack. New clients will be made aware of shallow upstream and can make use of this information. The sender must send all shallow commits that are sent in the following pack. It may send more shallow commits than necessary. upload-pack for example may choose to advertise no shallow commits if it knows in advance that the pack it's going to send contains no shallow commits. But upload-pack is the server, so we choose the cheaper way, send full .git/shallow and let the client deal with it. Smart HTTP is not affected by this patch. Shallow support on smart-http comes later separately. (*) A shallow commit is a commit that terminates the revision walker. It is usually put in .git/shallow in order to keep the revision walker from going out of bound because there is no guarantee that objects behind this commit is available. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-28upload-pack: ignore 'shallow' lines with unknown obj-idsLibravatar Michael Heemskerk1-2/+1
When the client sends a 'shallow' line for an object that the server does not have, the server currently dies with the error: "did not find object for shallow <obj-id>". The client may have truncated the history at the commit by fetching shallowly from a different server, or the commit may have been garbage collected by the server. In either case, this unknown commit is not relevant for calculating the pack that is to be sent and can be safely ignored, and it is not used when recomputing where the updated history of the client is cauterised. The documentation in technical/pack-protocol.txt has been updated to remove the restriction that "Clients MUST NOT mention an obj-id which it does not know exists on the server". This requirement is not realistic because clients cannot know whether an object has been garbage collected by the server. Signed-off-by: Michael Heemskerk <mheemskerk@atlassian.com> Reviewed-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-10-16Documentation/technical: convert plain text files to asciidocLibravatar Thomas Ackermann1-3/+4
These were not originally meant for asciidoc, but they are already so close. Mark them up in asciidoc. Signed-off-by: Thomas Ackermann <th.acker@arcor.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-18Doc: Improve shallow depth wordingLibravatar Philip Oakley1-2/+4
Avoid confusion in compound sentence about the start of the commit set and the depth measure. Use two sentences. Signed-off-by: Philip Oakley <philipoakley@iee.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-14pack-protocol: fix first-want separator in the examplesLibravatar Carlos Martín Nieto1-2/+2
When sending the "want" list, the capabilities list is separated from the obj-id by a SP instead of NUL as in the ref advertisement. The text is correct, but the examples wrongly show the separator as NUL. Fix the example so it uses SP. Signed-off-by: Carlos Martín Nieto <cmn@elego.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-03pack-protocol: document "ERR" lineLibravatar Nguyen Thai Ngoc Duy1-0/+7
Since a807328 (connect.c: add a way for git-daemon to pass an error back to client), git client recognizes "ERR" line and prints a friendly message to user if an error happens at server side. Document this. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-08Document the underlying protocol used by shallow repositories and --depth ↵Libravatar Alex Neronskiy1-22/+66
commands. Explain the exchange that occurs between a client and server when the client is requesting shallow history and/or is already using a shallow repository. Signed-off-by: Alex Neronskiy <zakmagnus@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-08Fix documentation of fetch-pack that implies that the client can disconnect ↵Libravatar Alex Neronskiy1-14/+15
after sending wants. Specify conditions under which the client can terminate the connection early. Previously, an unintended behavior was possible which could confuse servers. Based-on-patch-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Alex Neronskiy <zakmagnus@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-04pack-protocol.txt: fix pkt-line lengthsLibravatar Tay Ray Chuan1-4/+4
Previously, the lengths were 4-bytes short. Fix it such that the lengths reflect the total length of the pkt-line, as per spec. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-04pack-protocol.txt: fix spellingLibravatar Tay Ray Chuan1-1/+1
s/paramater/parameter/. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-01-31Fix typos in technical documentation.Libravatar Ralf Wildenhues1-5/+5
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-13Update packfile transfer protocol documentationLibravatar Scott Chacon1-41/+494
The current technical documentation for the packfile protocol is both sparse and incorrect. This documents the fetch-pack/upload-pack and send-pack/ receive-pack protocols much more fully. Add documentation from Shawn's upcoming http-protocol docs that is shared by the packfile protocol. protocol-common.txt describes ABNF notation amendments, refname rules and the packet line format. Add documentation on the various capabilities supported by the upload-pack and receive-pack protocols. protocol-capabilities.txt describes multi-ack, thin-pack, side-band[-64k], shallow, no-progress, include-tag, ofs-delta, delete-refs and report-status. Signed-off-by: Scott Chacon <schacon@gmail.com> Signed-off-by: Nanako Shiraishi <nanako3@lavabit.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2005-12-12Documentation: fix missing links to git(7)Libravatar Junio C Hamano1-0/+41
Also move pack protocol description to technical/. Signed-off-by: Junio C Hamano <junkio@cox.net>