From 77ccc5bbd1bd403abd5f552be7210073bea856a6 Mon Sep 17 00:00:00 2001 From: "Shawn O. Pearce" Date: Sat, 23 Dec 2006 02:33:35 -0500 Subject: Introduce new config option for mmap limit. Rather than hardcoding the maximum number of bytes which can be mmapped from pack files we should make this value configurable, allowing the end user to increase or decrease this limit on a per-repository basis depending on the size of the repository and the capabilities of their operating system. In general users should not need to manually tune such a low-level setting within the core code, but being able to artifically limit the number of bytes which we can mmap at once from pack files will make it easier to craft test cases for the new mmap sliding window implementation. Signed-off-by: Shawn O. Pearce Signed-off-by: Junio C Hamano --- Documentation/config.txt | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'Documentation/config.txt') diff --git a/Documentation/config.txt b/Documentation/config.txt index 178e0e1e20..28fe6942cf 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -118,6 +118,15 @@ core.legacyheaders:: database directly (where the "http://" and "rsync://" protocols count as direct access). +core.packedGitLimit:: + Maximum number of bytes to map simultaneously into memory + from pack files. If Git needs to access more than this many + bytes at once to complete an operation it will unmap existing + regions to reclaim virtual address space within the process. + Default is 256 MiB, which should be reasonable for all + users/operating systems, except on largest Git projects. + You probably do not need to adjust this value. + alias.*:: Command aliases for the gitlink:git[1] command wrapper - e.g. after defining "alias.last = cat-file commit HEAD", the invocation -- cgit v1.2.3 From 60bb8b1453e2f93d13e7bb44e8e46c085d2dd752 Mon Sep 17 00:00:00 2001 From: "Shawn O. Pearce" Date: Sat, 23 Dec 2006 02:34:28 -0500 Subject: Fully activate the sliding window pack access. This finally turns on the sliding window behavior for packfile data access by mapping limited size windows and chaining them under the packed_git->windows list. We consider a given byte offset to be within the window only if there would be at least 20 bytes (one hash worth of data) accessible after the requested offset. This range selection relates to the contract that use_pack() makes with its callers, allowing them to access one hash or one object header without needing to call use_pack() for every byte of data obtained. In the worst case scenario we will map the same page of data twice into memory: once at the end of one window and once again at the start of the next window. This duplicate page mapping will happen only when an object header or a delta base reference is spanned over the end of a window and is always limited to just one page of duplication, as no sane operating system will ever have a page size smaller than a hash. I am assuming that the possible wasted page of virtual address space is going to perform faster than the alternatives, which would be to copy the object header or ref delta into a temporary buffer prior to parsing, or to check the window range on every byte during header parsing. We may decide to revisit this decision in the future since this is just a gut instinct decision and has not actually been proven out by experimental testing. Signed-off-by: Shawn O. Pearce Signed-off-by: Junio C Hamano --- Documentation/config.txt | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 'Documentation/config.txt') diff --git a/Documentation/config.txt b/Documentation/config.txt index 28fe6942cf..d71653dc65 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -118,6 +118,17 @@ core.legacyheaders:: database directly (where the "http://" and "rsync://" protocols count as direct access). +core.packedGitWindowSize:: + Number of bytes of a pack file to map into memory in a + single mapping operation. Larger window sizes may allow + your system to process a smaller number of large pack files + more quickly. Smaller window sizes will negatively affect + performance due to increased calls to the opreating system's + memory manager, but may improve performance when accessing + a large number of large pack files. Default is 32 MiB, + which should be reasonable for all users/operating systems. + You probably do not need to adjust this value. + core.packedGitLimit:: Maximum number of bytes to map simultaneously into memory from pack files. If Git needs to access more than this many -- cgit v1.2.3 From eb92242f19e58de9c930220caf6bf1b83df54160 Mon Sep 17 00:00:00 2001 From: "Shawn O. Pearce" Date: Sat, 30 Dec 2006 22:13:43 -0500 Subject: Update packedGit config option documentation. Corrected minor typos and documented the new k/m/g suffix for core.packedGitWindowSize and core.packedGitLimit. [jc: with a minor markup fix.] Signed-off-by: Shawn O. Pearce Signed-off-by: Junio C Hamano --- Documentation/config.txt | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) (limited to 'Documentation/config.txt') diff --git a/Documentation/config.txt b/Documentation/config.txt index 744484b982..6c83829018 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -123,11 +123,13 @@ core.packedGitWindowSize:: single mapping operation. Larger window sizes may allow your system to process a smaller number of large pack files more quickly. Smaller window sizes will negatively affect - performance due to increased calls to the opreating system's + performance due to increased calls to the operating system's memory manager, but may improve performance when accessing a large number of large pack files. Default is 32 MiB, which should be reasonable for all users/operating systems. You probably do not need to adjust this value. ++ +Common unit suffixes of 'k', 'm', or 'g' are supported. core.packedGitLimit:: Maximum number of bytes to map simultaneously into memory @@ -135,8 +137,10 @@ core.packedGitLimit:: bytes at once to complete an operation it will unmap existing regions to reclaim virtual address space within the process. Default is 256 MiB, which should be reasonable for all - users/operating systems, except on largest Git projects. + users/operating systems, except on the largest projects. You probably do not need to adjust this value. ++ +Common unit suffixes of 'k', 'm', or 'g' are supported. alias.*:: Command aliases for the gitlink:git[1] command wrapper - e.g. -- cgit v1.2.3 From 22bac0ea528fd419cb833cab5de79a36fad91524 Mon Sep 17 00:00:00 2001 From: "Shawn O. Pearce" Date: Thu, 4 Jan 2007 22:28:08 -0500 Subject: Increase packedGit{Limit,WindowSize} on 64 bit systems. If we have a 64 bit address space we can easily afford to commit a larger amount of virtual address space to pack file access. So on these platforms we should increase the default settings of core.packedGit{Limit,WindowSize} to something that will better handle very large projects. Thanks to Andy Whitcroft for pointing out that we can safely increase these defaults on such systems. Signed-off-by: Shawn O. Pearce Signed-off-by: Junio C Hamano --- Documentation/config.txt | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) (limited to 'Documentation/config.txt') diff --git a/Documentation/config.txt b/Documentation/config.txt index 6c83829018..b24d9dff62 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -125,9 +125,12 @@ core.packedGitWindowSize:: more quickly. Smaller window sizes will negatively affect performance due to increased calls to the operating system's memory manager, but may improve performance when accessing - a large number of large pack files. Default is 32 MiB, - which should be reasonable for all users/operating systems. - You probably do not need to adjust this value. + a large number of large pack files. ++ +Default is 1 MiB if NO_MMAP was set at compile time, otherwise 32 +MiB on 32 bit platforms and 1 GiB on 64 bit platforms. This should +be reasonable for all users/operating systems. You probably do +not need to adjust this value. + Common unit suffixes of 'k', 'm', or 'g' are supported. @@ -136,9 +139,10 @@ core.packedGitLimit:: from pack files. If Git needs to access more than this many bytes at once to complete an operation it will unmap existing regions to reclaim virtual address space within the process. - Default is 256 MiB, which should be reasonable for all - users/operating systems, except on the largest projects. - You probably do not need to adjust this value. ++ +Default is 256 MiB on 32 bit platforms and 8 GiB on 64 bit platforms. +This should be reasonable for all users/operating systems, except on +the largest projects. You probably do not need to adjust this value. + Common unit suffixes of 'k', 'm', or 'g' are supported. -- cgit v1.2.3 From 5c94f87e6b17cab5d3b87998d6c907745cb6f5a4 Mon Sep 17 00:00:00 2001 From: Nicolas Pitre Date: Fri, 12 Jan 2007 16:01:46 -0500 Subject: use 'init' instead of 'init-db' for shipped docs and tools While 'init-db' still is and probably will always remain a valid git command for obvious backward compatibility reasons, it would be a good idea to move shipped tools and docs to using 'init' instead. Signed-off-by: Nicolas Pitre Signed-off-by: Junio C Hamano --- Documentation/config.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation/config.txt') diff --git a/Documentation/config.txt b/Documentation/config.txt index b4aae0d0ae..f7dba8977f 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -100,7 +100,7 @@ core.sharedRepository:: group-writable). When 'all' (or 'world' or 'everybody'), the repository will be readable by all users, additionally to being group-shareable. When 'umask' (or 'false'), git will use permissions - reported by umask(2). See gitlink:git-init-db[1]. False by default. + reported by umask(2). See gitlink:git-init[1]. False by default. core.warnAmbiguousRefs:: If true, git will warn you if the ref name you passed it is ambiguous -- cgit v1.2.3