summaryrefslogtreecommitdiff
path: root/Documentation/git-index-pack.txt
AgeCommit message (Collapse)AuthorFilesLines
2021-09-07index-pack: add --progress-title optionLibravatar Ævar Arnfjörð Bjarmason1-0/+6
Add a --progress-title option to index-pack, when data is piped into index-pack its progress is a proxy for whatever's feeding it data. This option will allow us to set a more relevant progress bar title in "git bundle unbundle", and is also used in my "bundle-uri" RFC patches[1] by a new caller in fetch-pack.c. The code change in cmd_index_pack() won't handle "--progress-title=xyz", only "--progress-title xyz", and the "(i+1)" style (as opposed to "i + 1") is a bit odd. Not using the "--long-option=value" style is inconsistent with existing long options handled by cmd_index_pack(), but makes the code that needs to call it better (two strvec_push(), instead of needing a strvec_pushf()). Since the option is internal-only the inconsistency shouldn't matter. I'm copying the pattern to handle it as-is from the handling of the existing "-o" option in the same function, see 9cf6d3357aa (Add git-index-pack utility, 2005-10-12) for its addition. That's a short option, but the code to implement the two is the same in functionality and style. Eventually we'd like to migrate all of this this to parse_options(), which would make these differences in behavior go away. 1. https://lore.kernel.org/git/RFC-cover-00.13-0000000000-20210805T150534Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-01Merge branch 'jt/transfer-fsck-across-packs'Libravatar Junio C Hamano1-1/+6
The approach to "fsck" the incoming objects in "index-pack" is attractive for performance reasons (we have them already in core, inflated and ready to be inspected), but fundamentally cannot be applied fully when we receive more than one pack stream, as a tree object in one pack may refer to a blob object in another pack as ".gitmodules", when we want to inspect blobs that are used as ".gitmodules" file, for example. Teach "index-pack" to emit objects that must be inspected later and check them in the calling "fetch-pack" process. * jt/transfer-fsck-across-packs: fetch-pack: print and use dangling .gitmodules fetch-pack: with packfile URIs, use index-pack arg http-fetch: allow custom index-pack args http: allow custom index-pack args
2021-02-22fetch-pack: print and use dangling .gitmodulesLibravatar Jonathan Tan1-1/+6
Teach index-pack to print dangling .gitmodules links after its "keep" or "pack" line instead of declaring an error, and teach fetch-pack to check such lines printed. This allows the tree side of the .gitmodules link to be in one packfile and the blob side to be in another without failing the fsck check, because it is now fetch-pack which checks such objects after all packfiles have been downloaded and indexed (and not index-pack on an individual packfile, as it is before this commit). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-25builtin/index-pack.c: write reverse indexesLibravatar Taylor Blau1-5/+13
Teach 'git index-pack' to optionally write and verify reverse index with '--[no-]rev-index', as well as respecting the 'pack.writeReverseIndex' configuration option. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-17Documentation: mark `--object-format=sha256` as experimentalLibravatar Martin Ågren1-0/+2
After eff45daab8 ("repository: enable SHA-256 support by default", 2020-07-29), vanilla builds of Git enable the user to run, e.g., git init --object-format=sha256 and hack away. This can be a good way to gain experience with the SHA-256 world, e.g., to find bugs that GIT_TEST_DEFAULT_HASH=sha256 make test doesn't spot. But it really is a separate world: Such SHA-256 repos will live entirely separate from the (by now fairly large) set of SHA-1 repos. Interacting across the border is possible in principle, e.g., through "diff + apply" (or "format-patch + am"), but even that has its limitations: Applying a SHA-256 diff in a SHA-1 repo works in the simple case, but if you need to resort to `-3`, you're out of luck. Similarly, "push + pull" should work, but you really will be operating mostly offset from the rest of the world. That might be ok by the time you initialize your repository, and it might be ok for several months after that, but there might come a day when you're starting to regret your use of `git init --object-format=sha256` and have dug yourself into a fairly deep hole. There are currently topics in flight to document our data formats and protocols regarding SHA-256 and in some cases (midx and commit-graph), we're considering adjusting how the file formats indicate which object format to use. Wherever `--object-format` is mentioned in our documentation, let's make it clear that using it with "sha256" is experimental. If we later need to explain why we can't handle data we generated back in 2020, we can always point to this paragraph we're adding here. By "include::"-ing a small blurb, we should be able to be consistent throughout the documentation and can eventually gradually tone down the severity of this text. One day, we might even use it to start phasing out `--object-format=sha1`, but let's not get ahead of ourselves... There's also `extensions.objectFormat`, but it's only mentioned three times. Twice where we're adding this new disclaimer and in the third spot we already have a "do not edit" warning. From there, interested readers should eventually find this new one that we're adding here. Because `GIT_DEFAULT_HASH` provides another entry point to this functionality, document the experimental nature of it too. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30Merge branch 'jb/doc-packfile-name' into masterLibravatar Junio C Hamano1-2/+2
Doc update. * jb/doc-packfile-name: pack-write/docs: update regarding pack naming
2020-07-22pack-write/docs: update regarding pack namingLibravatar Johannes Berg1-2/+2
The index-pack documentation explicitly states that the pack name is derived from the sorted list of object names, but since commit 1190a1acf800 ("pack-objects: name pack files after trailer hash") that isn't true anymore. Be less explicit in the docs as to what the exact output is, and just say that it's whatever goes into the pack name. Also update a comment on write_idx_file() since it no longer modifies the sha1 variable (it's const now anyway), as noted by Junio. Fixes: 1190a1acf800 ("pack-objects: name pack files after trailer hash") Signed-off-by: Johannes Berg <johannes@sipsolutions.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-19builtin/index-pack: add option to specify hash algorithmLibravatar brian m. carlson1-0/+8
git index-pack is usually run in a repository, but need not be. Since packs don't contains information on the algorithm in use, instead relying on context, add an option to index-pack to tell it which one we're using in case someone runs it outside of a repository. Since using --stdin necessarily implies a repository, don't allow specifying an object format if it's provided to prevent users from passing an option that won't work. Add documentation for this option. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-02doc: keep first level section header in upper caseLibravatar Nguyễn Thái Ngọc Duy1-2/+2
When formatted as a man page, 1st section header is always in upper case even if we write it otherwise. Make all 1st section headers uppercase to keep it close to the final output. This does affect html since case is kept there, but I still think it's a good idea to maintain a consistent style for 1st section headers. Some sections perhaps should become second sections instead, where case is kept, and for better organization. I will update if anyone has suggestions about this. While at there I also make some header more consistent (e.g. examples vs example) and fix a couple minor things here and there. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15index-pack: support checking objects but not linksLibravatar Jonathan Tan1-0/+3
The index-pack command currently supports the --check-self-contained-and-connected argument, for internal use only, that instructs it to only check for broken links and not broken objects. For partial clones, we need the inverse, so add a --fsck-objects argument that checks for broken objects and not broken links, also for internal use only. This will be used by fetch-pack in a subsequent patch. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-24index-pack: add --max-input-size=<size> optionLibravatar Jeff King1-0/+2
When receiving a pack-file, it can be useful to abort the `git index-pack`, if the pack-file is too big. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-28clone: open a shortcut for connectivity checkLibravatar Nguyễn Thái Ngọc Duy1-0/+3
In order to make sure the cloned repository is good, we run "rev-list --objects --not --all $new_refs" on the repository. This is expensive on large repositories. This patch attempts to mitigate the impact in this special case. In the "good" clone case, we only have one pack. If all of the following are met, we can be sure that all objects reachable from the new refs exist, which is the intention of running "rev-list ...": - all refs point to an object in the pack - there are no dangling pointers in any object in the pack - no objects in the pack point to objects outside the pack The second and third checks can be done with the help of index-pack as a slight variation of --strict check (which introduces a new condition for the shortcut: pack transfer must be used and the number of objects large enough to call index-pack). The first is checked in check_everything_connected after we get an "ok" from index-pack. "index-pack + new checks" is still faster than the current "index-pack + rev-list", which is the whole point of this patch. If any of the conditions fail, we fall back to the good old but expensive "rev-list ..". In that case it's even more expensive because we have to pay for the new checks in index-pack. But that should only happen when the other side is either buggy or malicious. Cloning linux-2.6 over file:// before after real 3m25.693s 2m53.050s user 5m2.037s 4m42.396s sys 0m13.750s 0m16.574s A more realistic test with ssh:// over wireless before after real 11m26.629s 10m4.213s user 5m43.196s 5m19.444s sys 0m35.812s 0m37.630s This shortcut is not applied to shallow clones, partly because shallow clones should have no more objects than a usual fetch and the cost of rev-list is acceptable, partly to avoid dealing with corner cases when grafting is involved. This shortcut does not apply to unpack-objects code path either because the number of objects must be small in order to trigger that code path. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-15The name of the hash function is "SHA-1", not "SHA1"Libravatar Thomas Ackermann1-1/+1
Use "SHA-1" instead of "SHA1" whenever we talk about the hash function. When used as a programming symbol, we keep "SHA1". Signed-off-by: Thomas Ackermann <th.acker@arcor.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-01Documentation: the name of the system is 'Git', not 'git'Libravatar Thomas Ackermann1-3/+3
Signed-off-by: Thomas Ackermann <th.acker@arcor.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-07index-pack: support multithreaded delta resolvingLibravatar Nguyễn Thái Ngọc Duy1-0/+10
This puts delta resolving on each base on a separate thread, one base cache per thread. Per-thread data is grouped in struct thread_local. When running with nr_threads == 1, no pthreads calls are made. The system essentially runs in non-thread mode. An experiment on a Xeon 24 core machine with git.git shows that performance does not increase proportional to the number of cores. So by default, we use maximum 3 cores. Some numbers with --threads from 1 to 16: 1..4 real 0m8.003s 0m5.307s 0m4.321s 0m3.830s user 0m7.720s 0m8.009s 0m8.133s 0m8.305s sys 0m0.224s 0m0.372s 0m0.360s 0m0.360s 5..8 real 0m3.727s 0m3.604s 0m3.332s 0m3.369s user 0m9.361s 0m9.817s 0m9.525s 0m9.769s sys 0m0.584s 0m0.624s 0m0.540s 0m0.560s 9..12 real 0m3.036s 0m3.139s 0m3.177s 0m2.961s user 0m8.977s 0m10.205s 0m9.737s 0m10.073s sys 0m0.596s 0m0.680s 0m0.684s 0m0.680s 13..16 real 0m2.985s 0m2.894s 0m2.975s 0m2.971s user 0m9.825s 0m10.573s 0m10.833s 0m11.361s sys 0m0.788s 0m0.732s 0m0.904s 0m1.016s On an Intel dual core and linux-2.6.git 1..4 real 2m37.789s 2m7.963s 2m0.920s 1m58.213s user 2m28.415s 2m52.325s 2m50.176s 2m41.187s sys 0m7.808s 0m11.181s 0m11.224s 0m10.731s Thanks Ramsay Jones for troubleshooting and support on MinGW platform. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-11doc: drop author/documentation sections from most pagesLibravatar Jeff King1-9/+0
The point of these sections is generally to: 1. Give credit where it is due. 2. Give the reader an idea of where to ask questions or file bug reports. But they don't do a good job of either case. For (1), they are out of date and incomplete. A much more accurate answer can be gotten through shortlog or blame. For (2), the correct contact point is generally git@vger, and even if you wanted to cc the contact point, the out-of-date and incomplete fields mean you're likely sending to somebody useless. So let's drop the fields entirely from all manpages except git(1) itself. We already point people to the mailing list for bug reports there, and we can update the Authors section to give credit to the major contributors and point to shortlog and blame for more information. Each page has a "This is part of git" footer, so people can follow that to the main git manpage.
2010-10-08Use angles for placeholders consistentlyLibravatar Štěpán Němec1-2/+2
Signed-off-by: Štěpán Němec <stepnem@gmail.com> Acked-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-18Documentation: describe --thin more accuratelyLibravatar Stephen Boyd1-8/+4
The description for --thin was misleading and downright wrong. Correct it with some inspiration from the description of index-pack's --fix-thin and some background information from Nicolas Pitre <nico@fluxnic.net>. Signed-off-by: Stephen Boyd <bebarino@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-01-10Documentation: spell 'git cmd' without dash throughoutLibravatar Thomas Rast1-4/+4
The documentation was quite inconsistent when spelling 'git cmd' if it only refers to the program, not to some specific invocation syntax: both 'git-cmd' and 'git cmd' spellings exist. The current trend goes towards dashless forms, and there is precedent in 647ac70 (git-svn.txt: stop using dash-form of commands., 2009-07-07) to actively eliminate the dashed variants. Replace 'git-cmd' with 'git cmd' throughout, except where git-shell, git-cvsserver, git-upload-pack, git-receive-pack, and git-upload-archive are concerned, because those really live in the $PATH.
2008-07-05manpages: italicize git command names (which were in teletype font)Libravatar Jonathan Nieder1-4/+4
The names of git commands are not meant to be entered at the commandline; they are just names. So we render them in italics, as is usual for command names in manpages. Using doit () { perl -e 'for (<>) { s/\`(git-[^\`.]*)\`/'\''\1'\''/g; print }' } for i in git*.txt config.txt diff*.txt blame*.txt fetch*.txt i18n.txt \ merge*.txt pretty*.txt pull*.txt rev*.txt urls*.txt do doit <"$i" >"$i+" && mv "$i+" "$i" done git diff . Signed-off-by: Jonathan Nieder <jrnieder@uchicago.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-01Documentation formatting and cleanupLibravatar Jonathan Nieder1-4/+4
Following what appears to be the predominant style, format names of commands and commandlines both as `teletype text`. While we're at it, add articles ("a" and "the") in some places, italicize the name of the command in the manual page synopsis line, and add a comma or two where it seems appropriate. Signed-off-by: Jonathan Nieder <jrnieder@uchicago.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-01Documentation: be consistent about "git-" versus "git "Libravatar Jonathan Nieder1-2/+2
Since the git-* commands are not installed in $(bindir), using "git-command <parameters>" in examples in the documentation is not a good idea. On the other hand, it is nice to be able to refer to each command using one hyphenated word. (There is no escaping it, anyway: man page names cannot have spaces in them.) This patch retains the dash in naming an operation, command, program, process, or action. Complete command lines that can be entered at a shell (i.e., without options omitted) are made to use the dashless form. The changes consist only of replacing some spaces with hyphens and vice versa. After a "s/ /-/g", the unpatched and patched versions are identical. Signed-off-by: Jonathan Nieder <jrnieder@uchicago.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-06documentation: move git(7) to git(1)Libravatar Christian Couder1-1/+1
As the "git" man page describes the "git" command at the end-user level, it seems better to move it to man section 1. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-28index-pack: introduce checking modeLibravatar Martin Koegler1-0/+3
Adds strict option, which bails out if the pack would introduces broken object or links in the repository. Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-06Documentation: rename gitlink macro to linkgitLibravatar Dan McGee1-5/+5
Between AsciiDoc 8.2.2 and 8.2.3, the following change was made to the stock Asciidoc configuration: @@ -149,7 +153,10 @@ # Inline macros. # Backslash prefix required for escape processing. # (?s) re flag for line spanning. -(?su)[\\]?(?P<name>\w(\w|-)*?):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])= + +# Explicit so they can be nested. +(?su)[\\]?(?P<name>(http|https|ftp|file|mailto|callto|image|link)):(?P<target>\S*?)(\[(?P<attrlist>.*?)\])= + # Anchor: [[[id]]]. Bibliographic anchor. (?su)[\\]?\[\[\[(?P<attrlist>[\w][\w-]*?)\]\]\]=anchor3 # Anchor: [[id,xreflabel]] This default regex now matches explicit values, and unfortunately in this case gitlink was being matched by just 'link', causing the wrong inline macro template to be applied. By renaming the macro, we can avoid being matched by the wrong regex. Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-06Documentation: fix "gitlink::foobar[s]"Libravatar Junio C Hamano1-1/+1
They should be spelled with a single colon. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-15Fix some typos, punctuation, missing words, minor markup.Libravatar Ralf Wildenhues1-1/+1
Signed-off-by: Lars Hjemli <hjemli@gmail.com> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-06-07War on whitespaceLibravatar Junio C Hamano1-1/+0
This uses "git-apply --whitespace=strip" to fix whitespace errors that have crept in to our source files over time. There are a few files that need to have trailing whitespaces (most notably, test vectors). The results still passes the test, and build result in Documentation/ area is unchanged. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-05-18Merge branch 'maint'Libravatar Junio C Hamano1-1/+3
* maint: Documentation: Reformatted SYNOPSIS for several commands Documentation: Added [verse] to SYNOPSIS where necessary
2007-05-18Documentation: Reformatted SYNOPSIS for several commandsLibravatar Matthias Kestenholz1-1/+3
Signed-off-by: Matthias Kestenholz <matthias@spinlock.ch> Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-04-19document --index-version for index-pack and pack-objectsLibravatar Nicolas Pitre1-0/+5
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-03remove .keep pack lock files when done with refs updateLibravatar Nicolas Pitre1-0/+11
This makes both git-fetch and git-push (fetch-pack and receive-pack) safe against a possible race with aparallel git-repack -a -d that could prune the new pack while it is not yet referenced, and remove the .keep file after refs have been updated. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-29Teach git-index-pack how to keep a pack file.Libravatar Shawn Pearce1-3/+21
To prevent a race condition between `index-pack --stdin` and `repack -a -d` where the repack deletes the newly created pack file before any refs are updated to reference objects contained within it we mark the pack file as one that should be kept. This removes it from the list of packs that `repack -a -d` will consider for removal. Callers such as `receive-pack` which want to invoke `index-pack` should use this new --keep option to prevent the newly created pack and index file pair from being deleted before they have finished any related ref updates. Only after all ref updates have been finished should the associated .keep file be removed. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-26add progress status to index-packLibravatar Nicolas Pitre1-2/+5
This is more interesting to look at when performing a big fetch. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-26make index-pack able to complete thin packs.Libravatar Nicolas Pitre1-1/+13
A new flag, --fix-thin, instructs git-index-pack to append any missing objects to a thin pack to make it self contained and indexable. Of course objects missing from the pack must be present elsewhere in the local repository. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-25enable index-pack streaming capabilityLibravatar Nicolas Pitre1-1/+7
A new flag, --stdin, allows for a pack to be received over a stream. When this flag is provided, the pack content is written to either the named pack file or directly to the object repository under the same name as produced by git-repack. The pack index is written as well with the corresponding base name, unless the index name is overriden with -o. With this patch, git-index-pack could be used instead of git-unpack-objects when fetching remote objects but only with non "thin" packs for now. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-12Add git-index-pack utilityLibravatar Sergey Vlasov1-0/+44
git-index-pack builds a pack index file for an existing packed archive. With this utility a packed archive which was transferred without the corresponding pack index can be added to objects/pack/ without repacking. Signed-off-by: Sergey Vlasov <vsu@altlinux.ru> Signed-off-by: Junio C Hamano <junkio@cox.net>