summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-04-13Merge branch 'ab/detox-config-gettext'Libravatar Junio C Hamano1-15/+1
The last remnant of gettext-poison has been removed. * ab/detox-config-gettext: config.c: remove last remnant of GIT_TEST_GETTEXT_POISON
2021-04-13Merge branch 'gk/gitweb-redacted-email'Libravatar Junio C Hamano2-7/+38
"gitweb" learned "e-mail privacy" feature to redact strings that look like e-mail addresses on various pages. * gk/gitweb-redacted-email: gitweb: add "e-mail privacy" feature to redact e-mail addresses
2021-04-13Merge branch 'cc/test-helper-bloom-usage-fix'Libravatar Junio C Hamano1-1/+1
Usage message fix for a test helper. * cc/test-helper-bloom-usage-fix: test-bloom: fix missing 'bloom' from usage string
2021-04-13Merge branch 'ab/send-email-validate-errors'Libravatar Junio C Hamano2-25/+55
Clean-up codepaths that implements "git send-email --validate" option and improves the message from it. * ab/send-email-validate-errors: git-send-email: improve --validate error output git-send-email: refactor duplicate $? checks into a function git-send-email: test full --validate output
2021-04-13Merge branch 'tb/precompose-prefix-simplify'Libravatar Junio C Hamano5-16/+29
Streamline the codepath to fix the UTF-8 encoding issues in the argv[] and the prefix on macOS. * tb/precompose-prefix-simplify: macOS: precompose startup_info->prefix precompose_utf8: make precompose_string_if_needed() public
2021-04-13Merge branch 'fm/user-manual-use-preface'Libravatar Junio C Hamano1-0/+3
Doc update to improve git.info * fm/user-manual-use-preface: user-manual.txt: assign preface an id and a title
2021-04-13Merge branch 'ab/perl-do-not-abuse-map'Libravatar Junio C Hamano1-2/+2
Perl critique. * ab/perl-do-not-abuse-map: git-send-email: replace "map" in void context with "for"
2021-04-13Merge branch 'tb/pack-preferred-tips-to-give-bitmap'Libravatar Junio C Hamano9-0/+142
A configuration variable has been added to force tips of certain refs to be given a reachability bitmap. * tb/pack-preferred-tips-to-give-bitmap: builtin/pack-objects.c: respect 'pack.preferBitmapTips' t/helper/test-bitmap.c: initial commit pack-bitmap: add 'test_bitmap_commits()' helper
2021-04-13Merge branch 'jk/ref-filter-segfault-fix'Libravatar Junio C Hamano2-1/+11
A NULL-dereference bug has been corrected in an error codepath in "git for-each-ref", "git branch --list" etc. * jk/ref-filter-segfault-fix: ref-filter: fix NULL check for parse object failure
2021-04-08gitweb: add "e-mail privacy" feature to redact e-mail addressesLibravatar Georgios Kontaxis2-7/+38
Gitweb extracts content from the Git log and makes it accessible over HTTP. As a result, e-mail addresses found in commits are exposed to web crawlers and they may not respect robots.txt. This can result in unsolicited messages. Introduce an 'email-privacy' feature which redacts e-mail addresses from the generated HTML content. Specifically, obscure addresses retrieved from the the author/committer and comment sections of the Git log. The feature is off by default. This feature does not prevent someone from downloading the unredacted commit log, e.g., by cloning the repository, and extracting information from it. It aims to hinder the low- effort, bulk collection of e-mail addresses by web crawlers. Signed-off-by: Georgios Kontaxis <geko1702+commits@99rst.org> Acked-by: Eric Wong <e@80x24.org> Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08The eighth batchLibravatar Junio C Hamano1-0/+18
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08Merge branch 'ab/make-tags-quiet'Libravatar Junio C Hamano1-4/+6
Generate [ec]tags under $(QUIET_GEN). * ab/make-tags-quiet: Makefile: add QUIET_GEN to "tags" and "TAGS" targets
2021-04-08Merge branch 'rs/daemon-sanitize-dir-sep'Libravatar Junio C Hamano1-4/+4
"git daemon" has been tightened against systems that take backslash as directory separator. * rs/daemon-sanitize-dir-sep: daemon: sanitize all directory separators
2021-04-08Merge branch 'en/ort-perf-batch-9'Libravatar Junio C Hamano4-15/+354
The ort merge backend has been optimized by skipping irrelevant renames. * en/ort-perf-batch-9: diffcore-rename: avoid doing basename comparisons for irrelevant sources merge-ort: skip rename detection entirely if possible merge-ort: use relevant_sources to filter possible rename sources merge-ort: precompute whether directory rename detection is needed merge-ort: introduce wrappers for alternate tree traversal merge-ort: add data structures for an alternate tree traversal merge-ort: precompute subset of sources for which we need rename detection diffcore-rename: enable filtering possible rename sources
2021-04-08Merge branch 'en/sequencer-edit-upon-conflict-fix'Libravatar Junio C Hamano4-17/+77
"git cherry-pick/revert" with or without "--[no-]edit" did not spawn the editor as expected (e.g. "revert --no-edit" after a conflict still asked to edit the message), which has been corrected. * en/sequencer-edit-upon-conflict-fix: sequencer: fix edit handling for cherry-pick and revert messages
2021-04-08Merge branch 'll/clone-reject-shallow'Libravatar Junio C Hamano10-6/+108
"git clone --reject-shallow" option fails the clone as soon as we notice that we are cloning from a shallow repository. * ll/clone-reject-shallow: builtin/clone.c: add --reject-shallow option
2021-04-08Merge branch 'tb/reverse-midx'Libravatar Junio C Hamano14-68/+733
An on-disk reverse-index to map the in-pack location of an object back to its object name across multiple packfiles is introduced. * tb/reverse-midx: midx.c: improve cache locality in midx_pack_order_cmp() pack-revindex: write multi-pack reverse indexes pack-write.c: extract 'write_rev_file_order' pack-revindex: read multi-pack reverse indexes Documentation/technical: describe multi-pack reverse indexes midx: make some functions non-static midx: keep track of the checksum midx: don't free midx_name early midx: allow marking a pack as preferred t/helper/test-read-midx.c: add '--show-objects' builtin/multi-pack-index.c: display usage on unrecognized command builtin/multi-pack-index.c: don't enter bogus cmd_mode builtin/multi-pack-index.c: split sub-commands builtin/multi-pack-index.c: define common usage with a macro builtin/multi-pack-index.c: don't handle 'progress' separately builtin/multi-pack-index.c: inline 'flags' with options
2021-04-08config.c: remove last remnant of GIT_TEST_GETTEXT_POISONLibravatar Ævar Arnfjörð Bjarmason1-15/+1
Remove a use of GIT_TEST_GETTEXT_POISON added in f276e2a4694 (config: improve error message for boolean config, 2021-02-11). This was simultaneously in-flight with my d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) which removed the rest of the GIT_TEST_GETTEXT_POISON code. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-07The seventh batchLibravatar Junio C Hamano1-0/+29
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-07Merge branch 'ab/fsck-api-cleanup'Libravatar Junio C Hamano7-216/+210
Fsck API clean-up. * ab/fsck-api-cleanup: fetch-pack: use new fsck API to printing dangling submodules fetch-pack: use file-scope static struct for fsck_options fetch-pack: don't needlessly copy fsck_options fsck.c: move gitmodules_{found,done} into fsck_options fsck.c: add an fsck_set_msg_type() API that takes enums fsck.c: pass along the fsck_msg_id in the fsck_error callback fsck.[ch]: move FOREACH_FSCK_MSG_ID & fsck_msg_id from *.c to *.h fsck.c: give "FOREACH_MSG_ID" a more specific name fsck.c: undefine temporary STR macro after use fsck.c: call parse_msg_type() early in fsck_set_msg_type() fsck.h: re-order and re-assign "enum fsck_msg_type" fsck.h: move FSCK_{FATAL,INFO,ERROR,WARN,IGNORE} into an enum fsck.c: refactor fsck_msg_type() to limit scope of "int msg_type" fsck.c: rename remaining fsck_msg_id "id" to "msg_id" fsck.c: remove (mostly) redundant append_msg_id() function fsck.c: rename variables in fsck_set_msg_type() for less confusion fsck.h: use "enum object_type" instead of "int" fsck.h: use designed initializers for FSCK_OPTIONS_{DEFAULT,STRICT} fsck.c: refactor and rename common config callback
2021-04-07Merge branch 'cc/downcase-opt-help'Libravatar Junio C Hamano2-5/+5
A few option description strings started with capital letters, which were corrected. * cc/downcase-opt-help: column, range-diff: downcase option description
2021-04-07Merge branch 'js/security-md'Libravatar Junio C Hamano3-0/+183
SECURITY.md that is facing individual contributors and end users has been introduced. Also a procedure to follow when preparing embargoed releases has been spelled out. * js/security-md: Document how we do embargoed releases SECURITY: describe how to report vulnerabilities
2021-04-07Merge branch 'ps/pack-bitmap-optim'Libravatar Junio C Hamano2-0/+15
Optimize "rev-list --use-bitmap-index --objects" corner case that uses negative tags as the stopping points. * ps/pack-bitmap-optim: pack-bitmap: avoid traversal of objects referenced by uninteresting tag
2021-04-07Merge branch 'zh/commit-trailer'Libravatar Junio C Hamano3-1/+347
"git commit" learned "--trailer <key>[=<value>]" option; together with the interpret-trailers command, this will make it easier to support custom trailers. * zh/commit-trailer: commit: add --trailer option
2021-04-07Merge branch 'js/cmake-vsbuild'Libravatar Junio C Hamano2-7/+24
CMake update for vsbuild. * js/cmake-vsbuild: cmake(install): include vcpkg dlls cmake: add a preparatory work-around to accommodate `vcpkg` cmake(install): fix double .exe suffixes cmake: support SKIP_DASHED_BUILT_INS
2021-04-07Merge branch 'ds/clarify-hashwrite'Libravatar Junio C Hamano1-15/+18
The hashwrite() API uses a buffering mechanism to avoid calling write(2) too frequently. This logic has been refactored to be easier to understand. * ds/clarify-hashwrite: csum-file: make hashwrite() more readable
2021-04-07Merge branch 'ah/plugleaks'Libravatar Junio C Hamano10-56/+74
Plug or annotate remaining leaks that trigger while running the very basic set of tests. * ah/plugleaks: transport: also free remote_refs in transport_disconnect() parse-options: don't leak alias help messages parse-options: convert bitfield values to use binary shift init-db: silence template_dir leak when converting to absolute path init: remove git_init_db_config() while fixing leaks worktree: fix leak in dwim_branch() clone: free or UNLEAK further pointers when finished reset: free instead of leaking unneeded ref symbolic-ref: don't leak shortened refname in check_symref()
2021-04-06git-send-email: improve --validate error outputLibravatar Ævar Arnfjörð Bjarmason2-11/+18
Improve the output we emit on --validate error to: * Say "FILE:LINE" instead of "FILE: LINE", to match "grep -n", compiler error messages etc. * Don't say "patch contains a" after just mentioning the filename, just leave it at "FILE:LINE: is longer than[...]. The "contains a" sounded like we were talking about the file in general, when we're actually checking it line-by-line. * Don't just say "rejected by sendemail-validate hook", but combine that with the system_or_msg() output to say what exit code the hook died with. I had an aborted attempt to make the line length checker note all lines that were longer than the limit. I didn't think that was worth the effort, but I've left in the testing change to check that we die as soon as we spot the first long line. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-06git-send-email: refactor duplicate $? checks into a functionLibravatar Ævar Arnfjörð Bjarmason1-17/+28
Refactor the duplicate checking of $? into a function. There's an outstanding series[1] wanting to add a third use of system() in this file, let's not copy this boilerplate anymore when that happens. 1. http://lore.kernel.org/git/87y2esg22j.fsf@evledraar.gmail.com Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-06git-send-email: test full --validate outputLibravatar Ævar Arnfjörð Bjarmason1-6/+18
Change the tests that grep substrings out of the output to use a full test_cmp, in preparation for improving the output. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-05test-bloom: fix missing 'bloom' from usage stringLibravatar Christian Couder1-1/+1
Like 'get_murmur3' and 'generate_filter', 'get_filter_for_commit' is a subcommand of `test-tool bloom` not of `test-tool` itself. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-05macOS: precompose startup_info->prefixLibravatar Torsten Bögershausen2-11/+19
The "prefix" was precomposed for macOS in commit 5c327502 (MacOS: precompose_argv_prefix(), 2021-02-03). However, this commit forgot to update "startup_info->prefix" after precomposing. Move the (possible) precomposition towards the end of setup_git_directory_gently(), so that precompose_string_if_needed() can use git_config_get_bool("core.precomposeunicode") correctly. Keep prefix, startup_info->prefix and GIT_PREFIX_ENVIRONMENT all in sync. And as a result, the prefix no longer needs to be precomposed in git.c Reported-by: Dmitry Torilov <d.torilov@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-05precompose_utf8: make precompose_string_if_needed() publicLibravatar Torsten Bögershausen3-5/+10
commit 5c327502 (MacOS: precompose_argv_prefix(), 2021-02-03) uses the function precompose_string_if_needed() internally. It is only used from precompose_argv_prefix() and therefore static in compat/precompose_utf8.c Expose this function, it will be used in the next commit. While there, allow passing a NULL pointer, which will return NULL. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-03user-manual.txt: assign preface an id and a titleLibravatar Firmin Martin1-0/+3
Two among the three warnings raised by "make git.info" are related to the fact that the preface has not id in user-manual.txt. user-manual.texi:15: warning: empty menu entry name in `* : idm4.' user-manual.texi:141: warning: @unnumbered missing argument This causes asciidoc creating an empty preface and an empty title tag in user-manual.xml which turns to be an empty node in user-manual.texi and git.info. Consequently, one can notice in user-manual.texi and git.info a node named "idm4" in the menu and the navigation bar. In emacs, the first entry of the menu in the git info page is even displayed as empty. This fix will name "Introduction" the preface and assign it an id. The result can be seen in the files: user-manual.{xml, texi, html, pdf} and git.info. For future reference, the diff between old and new user-manual.xml, user-manual.texi, git.info, user-manual.html (converted through html2markdown) and user-manual.pdf (converted through pdftotext) are attached. --- before/user-manual.xml 2021-04-04 03:58:47.758008722 +0200 +++ after/user-manual.xml 2021-04-04 03:56:40.520551163 +0200 @@ -7,8 +7,8 @@ <bookinfo> <title>Git User Manual</title> </bookinfo> -<preface> -<title></title> +<preface id="_introduction"> +<title>Introduction</title> <simpara>Git is a fast distributed revision control system.</simpara> <simpara>This manual is designed to be readable by someone with basic UNIX command-line skills, but no previous knowledge of Git.</simpara> --- before/user-manual.texi 2021-04-04 03:58:47.490005652 +0200 +++ after/user-manual.texi 2021-04-04 03:56:40.520551163 +0200 @@ -7,12 +7,12 @@ * Git: (git). A fast distributed revision control system @end direntry -@node Top, idm4, , (dir) +@node Top, Introduction, , (dir) @documentlanguage en @top Git User Manual @menu -* : idm4. +* Introduction:: * Repositories and Branches:: * Exploring Git history:: * Developing with Git:: @@ -137,8 +137,8 @@ @end detailmenu @end menu -@node idm4, Repositories and Branches, Top, Top -@unnumbered +@node Introduction, Repositories and Branches, Top, Top +@unnumbered Introduction Git is a fast distributed revision control system. @@ -178,7 +178,7 @@ Finally, see @ref{Notes and todo list for this manual} for ways that you can help make this manual more complete. -@node Repositories and Branches, Exploring Git history, idm4, Top +@node Repositories and Branches, Exploring Git history, Introduction, Top @chapter Repositories and Branches @menu --- before/git.info 2021-04-04 03:58:46.557994966 +0200 +++ after/git.info 2021-04-04 03:56:40.520551163 +0200 @@ -7,14 +7,14 @@ END-INFO-DIR-ENTRY -File: git.info, Node: Top, Next: idm4, Up: (dir) +File: git.info, Node: Top, Next: Introduction, Up: (dir) Git User Manual *************** * Menu: -* : idm4. +* Introduction:: * Repositories and Branches:: * Exploring Git history:: * Developing with Git:: @@ -137,7 +137,10 @@ -File: git.info, Node: idm4, Next: Repositories and Branches, Prev: Top, Up: Top +File: git.info, Node: Introduction, Next: Repositories and Branches, Prev: Top, Up: Top + +Introduction +************ Git is a fast distributed revision control system. @@ -174,7 +177,7 @@ that you can help make this manual more complete. -File: git.info, Node: Repositories and Branches, Next: Exploring Git history, Prev: idm4, Up: Top +File: git.info, Node: Repositories and Branches, Next: Exploring Git history, Prev: Introduction, Up: Top 1 Repositories and Branches *************************** @@ -5471,207 +5474,207 @@ ... Tag Table: Node: Top212 -Node: idm43164 -Node: Repositories and Branches4465 ... +Node: Introduction3179 +Node: Repositories and Branches4515 +Node: How to get a Git repository5128 ... End Tag Table --- before/user-manual.html.md 2021-04-04 05:20:55.378695854 +0200 +++ after/user-manual.html.md 2021-04-04 05:21:11.282850802 +0200 @@ -4,6 +4,8 @@ **Table of Contents** +Introduction + 1\. Repositories and Branches @@ -278,7 +280,7 @@ Todo list -# +# Introduction Git is a fast distributed revision control system. --- before/user-manual.pdf.txt 2021-04-04 05:28:20.367036836 +0200 +++ after/user-manual.pdf.txt 2021-04-04 05:30:01.680026312 +0200 @@ -487,6 +487,7 @@ vii +Introduction Git is a fast distributed revision control system. This manual is designed to be readable by someone with basic UNIX command-line skills, but no previous knowledge of Git. Chapter 1 and Chapter 2 explain how to fetch and study a project using git—read these chapters to learn how to build and test a Signed-off-by: Firmin Martin <firminmartin24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-02The sixth batchLibravatar Junio C Hamano1-0/+7
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-02Merge branch 'zh/format-patch-fractional-reroll-count'Libravatar Junio C Hamano6-12/+88
"git format-patch -v<n>" learned to allow a reroll count that is not an integer. * zh/format-patch-fractional-reroll-count: format-patch: allow a non-integral version numbers
2021-04-02Merge branch 'jh/simple-ipc'Libravatar Junio C Hamano21-52/+3315
A simple IPC interface gets introduced to build services like fsmonitor on top. * jh/simple-ipc: t0052: add simple-ipc tests and t/helper/test-simple-ipc tool simple-ipc: add Unix domain socket implementation unix-stream-server: create unix domain socket under lock unix-socket: disallow chdir() when creating unix domain sockets unix-socket: add backlog size option to unix_stream_listen() unix-socket: eliminate static unix_stream_socket() helper function simple-ipc: add win32 implementation simple-ipc: design documentation for new IPC mechanism pkt-line: add options argument to read_packetized_to_strbuf() pkt-line: add PACKET_READ_GENTLE_ON_READ_ERROR option pkt-line: do not issue flush packets in write_packetized_*() pkt-line: eliminate the need for static buffer in packet_write_gently()
2021-04-02Merge branch 'mt/parallel-checkout-part-1'Libravatar Junio C Hamano11-137/+277
Preparatory API changes for parallel checkout. * mt/parallel-checkout-part-1: entry: add checkout_entry_ca() taking preloaded conv_attrs entry: move conv_attrs lookup up to checkout_entry() entry: extract update_ce_after_write() from write_entry() entry: make fstat_output() and read_blob_entry() public entry: extract a header file for entry.c functions convert: add classification for conv_attrs struct convert: add get_stream_filter_ca() variant convert: add [async_]convert_to_working_tree_ca() variants convert: make convert_attrs() and convert structs public
2021-04-02git-send-email: replace "map" in void context with "for"Libravatar Ævar Arnfjörð Bjarmason1-2/+2
While using "map" instead of "for" or "map" instead of "grep" and vice-versa makes for interesting trivia questions when interviewing Perl programmers, it doesn't make for very readable code. Let's refactor this loop initially added in 8fd5bb7f44b (git send-email: add --annotate option, 2008-11-11) to be a for-loop instead. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01Makefile: add QUIET_GEN to "tags" and "TAGS" targetsLibravatar Ævar Arnfjörð Bjarmason1-4/+6
Don't show the very verbose $(FIND_SOURCE_FILES) command on every "make TAGS" invocation. Let's use "generate into temporary and rename to the final file, after seeing the command that generated the output finished successfully" pattern, to avoid leaving a file with an incorrect output generated by a failed command. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01midx.c: improve cache locality in midx_pack_order_cmp()Libravatar Jeff King1-26/+29
There is a lot of pointer dereferencing in the pre-image version of 'midx_pack_order_cmp()', which this patch gets rid of. Instead of comparing the pack preferred-ness and then the pack id, both of these checks are done at the same time by using the high-order bit of the pack id to represent whether it's preferred. Then the pack id and offset are compared as usual. This produces the same result so long as there are less than 2^31 packs, which seems like a likely assumption to make in practice. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01pack-revindex: write multi-pack reverse indexesLibravatar Taylor Blau2-0/+116
Implement the writing half of multi-pack reverse indexes. This is nothing more than the format describe a few patches ago, with a new set of helper functions that will be used to clear out stale .rev files corresponding to old MIDXs. Unfortunately, a very similar comparison function as the one implemented recently in pack-revindex.c is reimplemented here, this time accepting a MIDX-internal type. An effort to DRY these up would create more indirection and overhead than is necessary, so it isn't pursued here. Currently, there are no callers which pass the MIDX_WRITE_REV_INDEX flag, meaning that this is all dead code. But, that won't be the case for long, since subsequent patches will introduce the multi-pack bitmap, which will begin passing this field. (In midx.c:write_midx_internal(), the two adjacent if statements share a conditional, but are written separately since the first one will eventually also handle the MIDX_WRITE_BITMAP flag, which does not yet exist.) Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01pack-write.c: extract 'write_rev_file_order'Libravatar Taylor Blau2-11/+26
Existing callers provide the reverse index code with an array of 'struct pack_idx_entry *'s, which is then sorted by pack order (comparing the offsets of each object within the pack). Prepare for the multi-pack index to write a .rev file by providing a way to write the reverse index without an array of pack_idx_entry (which the MIDX code does not have). Instead, callers can invoke 'write_rev_index_positions()', which takes an array of uint32_t's. The ith entry in this array specifies the ith object's (in index order) position within the pack (in pack order). Expose this new function for use in a later patch, and rewrite the existing write_rev_file() in terms of this new function. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01pack-revindex: read multi-pack reverse indexesLibravatar Taylor Blau5-0/+199
Implement reading for multi-pack reverse indexes, as described in the previous patch. Note that these functions don't yet have any callers, and won't until multi-pack reachability bitmaps are introduced in a later patch series. In the meantime, this patch implements some of the infrastructure necessary to support multi-pack bitmaps. There are three new functions exposed by the revindex API: - load_midx_revindex(): loads the reverse index corresponding to the given multi-pack index. - midx_to_pack_pos() and pack_pos_to_midx(): these convert between the multi-pack index and pseudo-pack order. load_midx_revindex() and pack_pos_to_midx() are both relatively straightforward. load_midx_revindex() needs a few functions to be exposed from the midx API. One to get the checksum of a midx, and another to get the .rev's filename. Similar to recent changes in the packed_git struct, three new fields are added to the multi_pack_index struct: one to keep track of the size, one to keep track of the mmap'd pointer, and another to point past the header and at the reverse index's data. pack_pos_to_midx() simply reads the corresponding entry out of the table. midx_to_pack_pos() is the trickiest, since it needs to find an object's position in the psuedo-pack order, but that order can only be recovered in the .rev file itself. This mapping can be implemented with a binary search, but note that the thing we're binary searching over isn't an array of values, but rather a permuted order of those values. So, when comparing two items, it's helpful to keep in mind the difference. Instead of a traditional binary search, where you are comparing two things directly, here we're comparing a (pack, offset) tuple with an index into the multi-pack index. That index describes another (pack, offset) tuple, and it is _those_ two tuples that are compared. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01Documentation/technical: describe multi-pack reverse indexesLibravatar Taylor Blau1-0/+83
As a prerequisite to implementing multi-pack bitmaps, motivate and describe the format and ordering of the multi-pack reverse index. The subsequent patch will implement reading this format, and the patch after that will implement writing it while producing a multi-pack index. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01midx: make some functions non-staticLibravatar Taylor Blau2-2/+4
In a subsequent commit, pack-revindex.c will become responsible for sorting a list of objects in the "MIDX pack order" (which will be defined in the following patch). To do so, it will need to be know the pack identifier and offset within that pack for each object in the MIDX. The MIDX code already has functions for doing just that (nth_midxed_offset() and nth_midxed_pack_int_id()), but they are statically declared. Since there is no reason that they couldn't be exposed publicly, and because they are already doing exactly what the caller in pack-revindex.c will want, expose them publicly so that they can be reused there. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01midx: keep track of the checksumLibravatar Taylor Blau1-1/+2
write_midx_internal() uses a hashfile to write the multi-pack index, but discards its checksum. This makes sense, since nothing that takes place after writing the MIDX cares about its checksum. That is about to change in a subsequent patch, when the optional reverse index corresponding to the MIDX will want to include the MIDX's checksum. Store the checksum of the MIDX in preparation for that. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01midx: don't free midx_name earlyLibravatar Taylor Blau1-1/+0
A subsequent patch will need to refer back to 'midx_name' later on in the function. In fact, this variable is already free()'d later on, so this makes the later free() no longer redundant. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01midx: allow marking a pack as preferredLibravatar Taylor Blau7-18/+148
When multiple packs in the multi-pack index contain the same object, the MIDX machinery must make a choice about which pack it associates with that object. Prior to this patch, the lowest-ordered[1] pack was always selected. Pack selection for duplicate objects is relatively unimportant today, but it will become important for multi-pack bitmaps. This is because we can only invoke the pack-reuse mechanism when all of the bits for reused objects come from the reuse pack (in order to ensure that all reused deltas can find their base objects in the same pack). To encourage the pack selection process to prefer one pack over another (the pack to be preferred is the one a caller would like to later use as a reuse pack), introduce the concept of a "preferred pack". When provided, the MIDX code will always prefer an object found in a preferred pack over any other. No format changes are required to store the preferred pack, since it will be able to be inferred with a corresponding MIDX bitmap, by looking up the pack associated with the object in the first bit position (this ordering is described in detail in a subsequent commit). [1]: the ordering is specified by MIDX internals; for our purposes we can consider the "lowest ordered" pack to be "the one with the most-recent mtime. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-01builtin/clone.c: add --reject-shallow optionLibravatar Li Linchao10-6/+108
In some scenarios, users may want more history than the repository offered for cloning, which happens to be a shallow repository, can give them. But because users don't know it is a shallow repository until they download it to local, we may want to refuse to clone this kind of repository, without creating any unnecessary files. The '--depth=x' option cannot be used as a solution; the source may be deep enough to give us 'x' commits when cloned, but the user may later need to deepen the history to arbitrary depth. Teach '--reject-shallow' option to "git clone" to abort as soon as we find out that we are cloning from a shallow repository. Signed-off-by: Li Linchao <lilinchao@oschina.cn> Signed-off-by: Junio C Hamano <gitster@pobox.com>