From 4c2db93807f5ab65976a901b562e4bc8d69d40bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nguy=E1=BB=85n=20Th=C3=A1i=20Ng=E1=BB=8Dc=20Duy?= Date: Sat, 14 Apr 2018 17:34:59 +0200 Subject: read-cache.c: make $GIT_TEST_SPLIT_INDEX boolean MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit While at there, document about this special mode when running the test suite. Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano --- t/README | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 't') diff --git a/t/README b/t/README index 24ddebfabf..d5e0a3c786 100644 --- a/t/README +++ b/t/README @@ -293,6 +293,17 @@ and know what setup is needed for it. Or when you want to run everything up to a certain test. +Running tests with special setups +--------------------------------- + +The whole test suite could be run to test some special features +that cannot be easily covered by a few specific test cases. These +could be enabled by running the test suite with correct GIT_TEST_ +environment set. + +GIT_TEST_SPLIT_INDEX= forces split-index mode on the whole +test suite. Accept any boolean values that are accepted by git-config. + Naming Tests ------------ -- cgit v1.2.3 From 43fa44fa3b68e6570145126892e1e43380d7bb5a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nguy=E1=BB=85n=20Th=C3=A1i=20Ng=E1=BB=8Dc=20Duy?= Date: Sat, 14 Apr 2018 17:35:05 +0200 Subject: pack-objects: move in_pack out of struct object_entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Instead of using 8 bytes (on 64 bit arch) to store a pointer to a pack. Use an index instead since the number of packs should be relatively small. This limits the number of packs we can handle to 1k. Since we can't be sure people can never run into the situation where they have more than 1k pack files. Provide a fall back route for it. If we find out they have too many packs, the new in_pack_by_idx[] array (which has at most 1k elements) will not be used. Instead we allocate in_pack[] array that holds nr_objects elements. This is similar to how the optional in_pack_pos field is handled. The new simple test is just to make sure the too-many-packs code path is at least executed. The true test is running make test GIT_TEST_FULL_IN_PACK_ARRAY=1 to take advantage of other special case tests. Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano --- t/README | 5 +++++ t/t5300-pack-object.sh | 5 +++++ 2 files changed, 10 insertions(+) (limited to 't') diff --git a/t/README b/t/README index d5e0a3c786..a06a85bae8 100644 --- a/t/README +++ b/t/README @@ -304,6 +304,11 @@ environment set. GIT_TEST_SPLIT_INDEX= forces split-index mode on the whole test suite. Accept any boolean values that are accepted by git-config. +GIT_TEST_FULL_IN_PACK_ARRAY= exercises the uncommon +pack-objects code path where there are more than 1024 packs even if +the actual number of packs in repository is below this limit. Accept +any boolean values that are accepted by git-config. + Naming Tests ------------ diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh index 65ff60f2ee..54eff03851 100755 --- a/t/t5300-pack-object.sh +++ b/t/t5300-pack-object.sh @@ -457,6 +457,11 @@ test_expect_success !PTHREADS,C_LOCALE_OUTPUT 'pack-objects --threads=N or pack. grep -F "no threads support, ignoring pack.threads" err ' +test_expect_success 'pack-objects in too-many-packs mode' ' + GIT_TEST_FULL_IN_PACK_ARRAY=1 git repack -ad && + git fsck +' + # # WARNING! # -- cgit v1.2.3 From ac77d0c370dc5b23ac1ec4a13c754fe7ffa48564 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nguy=E1=BB=85n=20Th=C3=A1i=20Ng=E1=BB=8Dc=20Duy?= Date: Sat, 14 Apr 2018 17:35:10 +0200 Subject: pack-objects: shrink size field in struct object_entry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It's very very rare that an uncompressed object is larger than 4GB (partly because Git does not handle those large files very well to begin with). Let's optimize it for the common case where object size is smaller than this limit. Shrink size field down to 31 bits and one overflow bit. If the size is too large, we read it back from disk. As noted in the previous patch, we need to return the delta size instead of canonical size when the to-be-reused object entry type is a delta instead of a canonical one. Add two compare helpers that can take advantage of the overflow bit (e.g. if the file is 4GB+, chances are it's already larger than core.bigFileThreshold and there's no point in comparing the actual value). Another note about oe_get_size_slow(). This function MUST be thread safe because SIZE() macro is used inside try_delta() which may run in parallel. Outside parallel code, no-contention locking should be dirt cheap (or insignificant compared to i/o access anyway). To exercise this code, it's best to run the test suite with something like make test GIT_TEST_OE_SIZE=4 which forces this code on all objects larger than 3 bytes. Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano --- t/README | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 't') diff --git a/t/README b/t/README index a06a85bae8..8373a27fea 100644 --- a/t/README +++ b/t/README @@ -309,6 +309,12 @@ pack-objects code path where there are more than 1024 packs even if the actual number of packs in repository is below this limit. Accept any boolean values that are accepted by git-config. +GIT_TEST_OE_SIZE= exercises the uncommon pack-objects code path +where we do not cache object size in memory and read it from existing +packs on demand. This normally only happens when the object size is +over 2GB. This variable forces the code path on any object larger than + bytes. + Naming Tests ------------ -- cgit v1.2.3