diff options
Diffstat (limited to 'Documentation/technical/pack-format.txt')
| -rw-r--r-- | Documentation/technical/pack-format.txt | 83 |
1 files changed, 66 insertions, 17 deletions
diff --git a/Documentation/technical/pack-format.txt b/Documentation/technical/pack-format.txt index d3a142c652..1faa949bf6 100644 --- a/Documentation/technical/pack-format.txt +++ b/Documentation/technical/pack-format.txt @@ -1,6 +1,12 @@ Git pack format =============== +== Checksums and object IDs + +In a repository using the traditional SHA-1, pack checksums, index checksums, +and object IDs (object names) mentioned below are all computed using SHA-1. +Similarly, in SHA-256 repositories, these values are computed using SHA-256. + == pack-*.pack files have the following format: - A header appears at the beginning and consists of the following: @@ -26,7 +32,7 @@ Git pack format (deltified representation) n-byte type and length (3-bit type, (n-1)*7+4-bit length) - 20-byte base object name if OBJ_REF_DELTA or a negative relative + base object name if OBJ_REF_DELTA or a negative relative offset from the delta object's position in the pack if this is an OBJ_OFS_DELTA object compressed delta data @@ -34,7 +40,7 @@ Git pack format Observation: length of each object is encoded in a variable length format and is not constrained to 32-bit or anything. - - The trailer records 20-byte SHA-1 checksum of all of the above. + - The trailer records a pack checksum of all of the above. === Object types @@ -49,6 +55,18 @@ Valid object types are: Type 5 is reserved for future expansion. Type 0 is invalid. +=== Size encoding + +This document uses the following "size encoding" of non-negative +integers: From each byte, the seven least significant bits are +used to form the resulting integer. As long as the most significant +bit is 1, this process continues; the byte with MSB 0 provides the +last seven bits. The seven-bit chunks are concatenated. Later +values are more significant. + +This size encoding should not be confused with the "offset encoding", +which is also used in this document. + === Deltified representation Conceptually there are only four object types: commit, tree, tag and @@ -58,8 +76,8 @@ ofs-delta and ref-delta, which is only valid in a pack file. Both ofs-delta and ref-delta store the "delta" to be applied to another object (called 'base object') to reconstruct the object. The -difference between them is, ref-delta directly encodes 20-byte base -object name. If the base object is in the same pack, ofs-delta encodes +difference between them is, ref-delta directly encodes base object +name. If the base object is in the same pack, ofs-delta encodes the offset of the base object in the pack instead. The base object could also be deltified if it's in the same pack. @@ -67,7 +85,10 @@ Ref-delta can also refer to an object outside the pack (i.e. the so-called "thin pack"). When stored on disk however, the pack should be self contained to avoid cyclic dependency. -The delta data is a sequence of instructions to reconstruct an object +The delta data starts with the size of the base object and the +size of the object to be reconstructed. These sizes are +encoded using the size encoding from above. The remainder of +the delta data is a sequence of instructions to reconstruct the object from the base object. If the base object is deltified, it must be converted to canonical form first. Each instruction appends more and more data to the target object until it's complete. There are two @@ -143,14 +164,14 @@ This is the instruction reserved for future expansion. object is stored in the packfile as the offset from the beginning. - 20-byte object name. + one object name of the appropriate size. - The file is concluded with a trailer: - A copy of the 20-byte SHA-1 checksum at the end of - corresponding packfile. + A copy of the pack checksum at the end of the corresponding + packfile. - 20-byte SHA-1-checksum of all of the above. + Index checksum of all of the above. Pack Idx file: @@ -198,7 +219,7 @@ Pack file entry: <+ If it is not DELTA, then deflated bytes (the size above is the size before compression). If it is REF_DELTA, then - 20-byte base object name SHA-1 (the size above is the + base object name (the size above is the size of the delta data that follows). delta data, deflated. If it is OFS_DELTA, then @@ -227,9 +248,9 @@ Pack file entry: <+ - A 256-entry fan-out table just like v1. - - A table of sorted 20-byte SHA-1 object names. These are - packed together without offset values to reduce the cache - footprint of the binary search for a specific object name. + - A table of sorted object names. These are packed together + without offset values to reduce the cache footprint of the + binary search for a specific object name. - A table of 4-byte CRC32 values of the packed object data. This is new in v2 so compressed data can be copied directly @@ -248,10 +269,30 @@ Pack file entry: <+ - The same trailer as a v1 pack file: - A copy of the 20-byte SHA-1 checksum at the end of + A copy of the pack checksum at the end of corresponding packfile. - 20-byte SHA-1-checksum of all of the above. + Index checksum of all of the above. + +== pack-*.rev files have the format: + + - A 4-byte magic number '0x52494458' ('RIDX'). + + - A 4-byte version identifier (= 1). + + - A 4-byte hash function identifier (= 1 for SHA-1, 2 for SHA-256). + + - A table of index positions (one per packed object, num_objects in + total, each a 4-byte unsigned integer in network order), sorted by + their corresponding offsets in the packfile. + + - A trailer, containing a: + + checksum of the corresponding packfile, and + + a checksum of all of the above. + +All 4-byte numbers are in network order. == multi-pack-index (MIDX) files have the following format: @@ -273,7 +314,12 @@ HEADER: Git only writes or recognizes version 1. 1-byte Object Id Version - Git only writes or recognizes version 1 (SHA1). + We infer the length of object IDs (OIDs) from this value: + 1 => SHA-1 + 2 => SHA-256 + If the hash type does not match the repository's hash algorithm, + the multi-pack-index file should be ignored with a warning + presented to the user. 1-byte number of "chunks" @@ -290,6 +336,9 @@ CHUNK LOOKUP: (Chunks are provided in file-order, so you can infer the length using the next chunk position if necessary.) + The CHUNK LOOKUP matches the table of contents from + link:technical/chunk-format.html[the chunk-based file format]. + The remaining data in the body is described one chunk at a time, and these chunks may be given in any order. Chunks are required unless otherwise specified. @@ -329,4 +378,4 @@ CHUNK DATA: TRAILER: - 20-byte SHA1-checksum of the above contents. + Index checksum of the above contents. |