summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--Documentation/config.txt138
-rw-r--r--fetch-pack.c32
-rwxr-xr-xt/t5504-fetch-receive-strict.sh126
3 files changed, 255 insertions, 41 deletions
diff --git a/Documentation/config.txt b/Documentation/config.txt
index fd8d27e761..c0eb78e344 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1502,10 +1502,19 @@ fetch.recurseSubmodules::
fetch.fsckObjects::
If it is set to true, git-fetch-pack will check all fetched
- objects. It will abort in the case of a malformed object or a
- broken link. The result of an abort are only dangling objects.
- Defaults to false. If not set, the value of `transfer.fsckObjects`
- is used instead.
+ objects. See `transfer.fsckObjects` for what's
+ checked. Defaults to false. If not set, the value of
+ `transfer.fsckObjects` is used instead.
+
+fetch.fsck.<msg-id>::
+ Acts like `fsck.<msg-id>`, but is used by
+ linkgit:git-fetch-pack[1] instead of linkgit:git-fsck[1]. See
+ the `fsck.<msg-id>` documentation for details.
+
+fetch.fsck.skipList::
+ Acts like `fsck.skipList`, but is used by
+ linkgit:git-fetch-pack[1] instead of linkgit:git-fsck[1]. See
+ the `fsck.skipList` documentation for details.
fetch.unpackLimit::
If the number of objects fetched over the Git native
@@ -1644,15 +1653,42 @@ filter.<driver>.smudge::
linkgit:gitattributes[5] for details.
fsck.<msg-id>::
- Allows overriding the message type (error, warn or ignore) of a
- specific message ID such as `missingEmail`.
-+
-For convenience, fsck prefixes the error/warning with the message ID,
-e.g. "missingEmail: invalid author/committer line - missing email" means
-that setting `fsck.missingEmail = ignore` will hide that issue.
-+
-This feature is intended to support working with legacy repositories
-which cannot be repaired without disruptive changes.
+ During fsck git may find issues with legacy data which
+ wouldn't be generated by current versions of git, and which
+ wouldn't be sent over the wire if `transfer.fsckObjects` was
+ set. This feature is intended to support working with legacy
+ repositories containing such data.
++
+Setting `fsck.<msg-id>` will be picked up by linkgit:git-fsck[1], but
+to accept pushes of such data set `receive.fsck.<msg-id>` instead, or
+to clone or fetch it set `fetch.fsck.<msg-id>`.
++
+The rest of the documentation discusses `fsck.*` for brevity, but the
+same applies for the corresponding `receive.fsck.*` and
+`fetch.<msg-id>.*`. variables.
++
+Unlike variables like `color.ui` and `core.editor` the
+`receive.fsck.<msg-id>` and `fetch.fsck.<msg-id>` variables will not
+fall back on the `fsck.<msg-id>` configuration if they aren't set. To
+uniformly configure the same fsck settings in different circumstances
+all three of them they must all set to the same values.
++
+When `fsck.<msg-id>` is set, errors can be switched to warnings and
+vice versa by configuring the `fsck.<msg-id>` setting where the
+`<msg-id>` is the fsck message ID and the value is one of `error`,
+`warn` or `ignore`. For convenience, fsck prefixes the error/warning
+with the message ID, e.g. "missingEmail: invalid author/committer line
+- missing email" means that setting `fsck.missingEmail = ignore` will
+hide that issue.
++
+In general, it is better to enumerate existing objects with problems
+with `fsck.skipList`, instead of listing the kind of breakages these
+problematic objects share to be ignored, as doing the latter will
+allow new instances of the same breakages go unnoticed.
++
+Setting an unknown `fsck.<msg-id>` value will cause fsck to die, but
+doing the same for `receive.fsck.<msg-id>` and `fetch.fsck.<msg-id>`
+will only cause git to warn.
fsck.skipList::
The path to a sorted list of object names (i.e. one SHA-1 per
@@ -1661,6 +1697,15 @@ fsck.skipList::
should be accepted despite early commits containing errors that
can be safely ignored such as invalid committer email addresses.
Note: corrupt objects cannot be skipped with this setting.
++
+Like `fsck.<msg-id>` this variable has corresponding
+`receive.fsck.skipList` and `fetch.fsck.skipList` variants.
++
+Unlike variables like `color.ui` and `core.editor` the
+`receive.fsck.skipList` and `fetch.fsck.skipList` variables will not
+fall back on the `fsck.skipList` configuration if they aren't set. To
+uniformly configure the same fsck settings in different circumstances
+all three of them they must all set to the same values.
gc.aggressiveDepth::
The depth parameter used in the delta compression
@@ -2947,32 +2992,21 @@ receive.certNonceSlop::
receive.fsckObjects::
If it is set to true, git-receive-pack will check all received
- objects. It will abort in the case of a malformed object or a
- broken link. The result of an abort are only dangling objects.
- Defaults to false. If not set, the value of `transfer.fsckObjects`
- is used instead.
+ objects. See `transfer.fsckObjects` for what's checked.
+ Defaults to false. If not set, the value of
+ `transfer.fsckObjects` is used instead.
receive.fsck.<msg-id>::
- When `receive.fsckObjects` is set to true, errors can be switched
- to warnings and vice versa by configuring the `receive.fsck.<msg-id>`
- setting where the `<msg-id>` is the fsck message ID and the value
- is one of `error`, `warn` or `ignore`. For convenience, fsck prefixes
- the error/warning with the message ID, e.g. "missingEmail: invalid
- author/committer line - missing email" means that setting
- `receive.fsck.missingEmail = ignore` will hide that issue.
-+
-This feature is intended to support working with legacy repositories
-which would not pass pushing when `receive.fsckObjects = true`, allowing
-the host to accept repositories with certain known issues but still catch
-other issues.
+ Acts like `fsck.<msg-id>`, but is used by
+ linkgit:git-receive-pack[1] instead of
+ linkgit:git-fsck[1]. See the `fsck.<msg-id>` documentation for
+ details.
receive.fsck.skipList::
- The path to a sorted list of object names (i.e. one SHA-1 per
- line) that are known to be broken in a non-fatal way and should
- be ignored. This feature is useful when an established project
- should be accepted despite early commits containing errors that
- can be safely ignored such as invalid committer email addresses.
- Note: corrupt objects cannot be skipped with this setting.
+ Acts like `fsck.skipList`, but is used by
+ linkgit:git-receive-pack[1] instead of
+ linkgit:git-fsck[1]. See the `fsck.skipList` documentation for
+ details.
receive.keepAlive::
After receiving the pack from the client, `receive-pack` may
@@ -3447,6 +3481,40 @@ transfer.fsckObjects::
When `fetch.fsckObjects` or `receive.fsckObjects` are
not set, the value of this variable is used instead.
Defaults to false.
++
+When set, the fetch or receive will abort in the case of a malformed
+object or a link to a nonexistent object. In addition, various other
+issues are checked for, including legacy issues (see `fsck.<msg-id>`),
+and potential security issues like the existence of a `.GIT` directory
+or a malicious `.gitmodules` file (see the release notes for v2.2.1
+and v2.17.1 for details). Other sanity and security checks may be
+added in future releases.
++
+On the receiving side, failing fsckObjects will make those objects
+unreachable, see "QUARANTINE ENVIRONMENT" in
+linkgit:git-receive-pack[1]. On the fetch side, malformed objects will
+instead be left unreferenced in the repository.
++
+Due to the non-quarantine nature of the `fetch.fsckObjects`
+implementation it can not be relied upon to leave the object store
+clean like `receive.fsckObjects` can.
++
+As objects are unpacked they're written to the object store, so there
+can be cases where malicious objects get introduced even though the
+"fetch" failed, only to have a subsequent "fetch" succeed because only
+new incoming objects are checked, not those that have already been
+written to the object store. That difference in behavior should not be
+relied upon. In the future, such objects may be quarantined for
+"fetch" as well.
++
+For now, the paranoid need to find some way to emulate the quarantine
+environment if they'd like the same protection as "push". E.g. in the
+case of an internal mirror do the mirroring in two steps, one to fetch
+the untrusted objects, and then do a second "push" (which will use the
+quarantine) to another internal repo, and have internal clients
+consume this pushed-to repository, or embargo internal fetches and
+only allow them once a full "fsck" has run (and no new fetches have
+happened in the meantime).
transfer.hideRefs::
String(s) `receive-pack` and `upload-pack` use to decide which
diff --git a/fetch-pack.c b/fetch-pack.c
index f80a7acdf3..88a078e9be 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -21,6 +21,7 @@
#include "object-store.h"
#include "connected.h"
#include "fetch-negotiator.h"
+#include "fsck.h"
static int transfer_unpack_limit = -1;
static int fetch_unpack_limit = -1;
@@ -36,6 +37,7 @@ static int server_supports_filtering;
static struct lock_file shallow_lock;
static const char *alternate_shallow_file;
static char *negotiation_algorithm;
+static struct strbuf fsck_msg_types = STRBUF_INIT;
/* Remember to update object flag allocation in object.h */
#define COMPLETE (1U << 0)
@@ -867,7 +869,8 @@ static int get_pack(struct fetch_pack_args *args,
*/
argv_array_push(&cmd.args, "--fsck-objects");
else
- argv_array_push(&cmd.args, "--strict");
+ argv_array_pushf(&cmd.args, "--strict%s",
+ fsck_msg_types.buf);
}
cmd.in = demux.out;
@@ -1401,6 +1404,31 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
return ref;
}
+static int fetch_pack_config_cb(const char *var, const char *value, void *cb)
+{
+ if (strcmp(var, "fetch.fsck.skiplist") == 0) {
+ const char *path;
+
+ if (git_config_pathname(&path, var, value))
+ return 1;
+ strbuf_addf(&fsck_msg_types, "%cskiplist=%s",
+ fsck_msg_types.len ? ',' : '=', path);
+ free((char *)path);
+ return 0;
+ }
+
+ if (skip_prefix(var, "fetch.fsck.", &var)) {
+ if (is_valid_msg_type(var, value))
+ strbuf_addf(&fsck_msg_types, "%c%s=%s",
+ fsck_msg_types.len ? ',' : '=', var, value);
+ else
+ warning("Skipping unknown msg id '%s'", var);
+ return 0;
+ }
+
+ return git_default_config(var, value, cb);
+}
+
static void fetch_pack_config(void)
{
git_config_get_int("fetch.unpacklimit", &fetch_unpack_limit);
@@ -1411,7 +1439,7 @@ static void fetch_pack_config(void)
git_config_get_string("fetch.negotiationalgorithm",
&negotiation_algorithm);
- git_config(git_default_config, NULL);
+ git_config(fetch_pack_config_cb, NULL);
}
static void fetch_pack_setup(void)
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index 49d3621a92..62f3569891 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -3,13 +3,16 @@
test_description='fetch/receive strict mode'
. ./test-lib.sh
-test_expect_success setup '
+test_expect_success 'setup and inject "corrupt or missing" object' '
echo hello >greetings &&
git add greetings &&
git commit -m greetings &&
S=$(git rev-parse :greetings | sed -e "s|^..|&/|") &&
X=$(echo bye | git hash-object -w --stdin | sed -e "s|^..|&/|") &&
+ echo $S >S &&
+ echo $X >X &&
+ cp .git/objects/$S .git/objects/$S.back &&
mv -f .git/objects/$X .git/objects/$S &&
test_must_fail git fsck
@@ -115,6 +118,13 @@ test_expect_success 'push with transfer.fsckobjects' '
test_cmp exp act
'
+test_expect_success 'repair the "corrupt or missing" object' '
+ mv -f .git/objects/$(cat S) .git/objects/$(cat X) &&
+ mv .git/objects/$(cat S).back .git/objects/$(cat S) &&
+ rm -rf .git/objects/$(cat X) &&
+ git fsck
+'
+
cat >bogus-commit <<EOF
tree $EMPTY_TREE
author Bugs Bunny 1234567890 +0000
@@ -123,6 +133,14 @@ committer Bugs Bunny <bugs@bun.ni> 1234567890 +0000
This commit object intentionally broken
EOF
+test_expect_success 'fsck with invalid or bogus skipList input' '
+ git -c fsck.skipList=/dev/null -c fsck.missingEmail=ignore fsck &&
+ test_must_fail git -c fsck.skipList=does-not-exist -c fsck.missingEmail=ignore fsck 2>err &&
+ test_i18ngrep "Could not open skip list: does-not-exist" err &&
+ test_must_fail git -c fsck.skipList=.git/config -c fsck.missingEmail=ignore fsck 2>err &&
+ test_i18ngrep "Invalid SHA-1: \[core\]" err
+'
+
test_expect_success 'push with receive.fsck.skipList' '
commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
git push . $commit:refs/heads/bogus &&
@@ -130,11 +148,61 @@ test_expect_success 'push with receive.fsck.skipList' '
git init dst &&
git --git-dir=dst/.git config receive.fsckObjects true &&
test_must_fail git push --porcelain dst bogus &&
- git --git-dir=dst/.git config receive.fsck.skipList SKIP &&
echo $commit >dst/.git/SKIP &&
+
+ # receive.fsck.* does not fall back on fsck.*
+ git --git-dir=dst/.git config fsck.skipList SKIP &&
+ test_must_fail git push --porcelain dst bogus &&
+
+ # Invalid and/or bogus skipList input
+ git --git-dir=dst/.git config receive.fsck.skipList /dev/null &&
+ test_must_fail git push --porcelain dst bogus &&
+ git --git-dir=dst/.git config receive.fsck.skipList does-not-exist &&
+ test_must_fail git push --porcelain dst bogus 2>err &&
+ test_i18ngrep "Could not open skip list: does-not-exist" err &&
+ git --git-dir=dst/.git config receive.fsck.skipList config &&
+ test_must_fail git push --porcelain dst bogus 2>err &&
+ test_i18ngrep "Invalid SHA-1: \[core\]" err &&
+
+ git --git-dir=dst/.git config receive.fsck.skipList SKIP &&
git push --porcelain dst bogus
'
+test_expect_success 'fetch with fetch.fsck.skipList' '
+ commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
+ refspec=refs/heads/bogus:refs/heads/bogus &&
+ git push . $commit:refs/heads/bogus &&
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckObjects true &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+ git --git-dir=dst/.git config fetch.fsck.skipList /dev/null &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+ echo $commit >dst/.git/SKIP &&
+
+ # fetch.fsck.* does not fall back on fsck.*
+ git --git-dir=dst/.git config fsck.skipList dst/.git/SKIP &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+
+ # Invalid and/or bogus skipList input
+ git --git-dir=dst/.git config fetch.fsck.skipList /dev/null &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+ git --git-dir=dst/.git config fetch.fsck.skipList does-not-exist &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec 2>err &&
+ test_i18ngrep "Could not open skip list: does-not-exist" err &&
+ git --git-dir=dst/.git config fetch.fsck.skipList dst/.git/config &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec 2>err &&
+ test_i18ngrep "Invalid SHA-1: \[core\]" err &&
+
+ git --git-dir=dst/.git config fetch.fsck.skipList dst/.git/SKIP &&
+ git --git-dir=dst/.git fetch "file://$(pwd)" $refspec
+'
+
+test_expect_success 'fsck.<unknownmsg-id> dies' '
+ test_must_fail git -c fsck.whatEver=ignore fsck 2>err &&
+ test_i18ngrep "Unhandled message id: whatever" err
+'
+
test_expect_success 'push with receive.fsck.missingEmail=warn' '
commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
git push . $commit:refs/heads/bogus &&
@@ -142,19 +210,58 @@ test_expect_success 'push with receive.fsck.missingEmail=warn' '
git init dst &&
git --git-dir=dst/.git config receive.fsckobjects true &&
test_must_fail git push --porcelain dst bogus &&
+
+ # receive.fsck.<msg-id> does not fall back on fsck.<msg-id>
+ git --git-dir=dst/.git config fsck.missingEmail warn &&
+ test_must_fail git push --porcelain dst bogus &&
+
+ # receive.fsck.<unknownmsg-id> warns
+ git --git-dir=dst/.git config \
+ receive.fsck.whatEver error &&
+
git --git-dir=dst/.git config \
receive.fsck.missingEmail warn &&
git push --porcelain dst bogus >act 2>&1 &&
grep "missingEmail" act &&
+ test_i18ngrep "Skipping unknown msg id.*whatever" act &&
git --git-dir=dst/.git branch -D bogus &&
git --git-dir=dst/.git config --add \
receive.fsck.missingEmail ignore &&
- git --git-dir=dst/.git config --add \
- receive.fsck.badDate warn &&
git push --porcelain dst bogus >act 2>&1 &&
! grep "missingEmail" act
'
+test_expect_success 'fetch with fetch.fsck.missingEmail=warn' '
+ commit="$(git hash-object -t commit -w --stdin <bogus-commit)" &&
+ refspec=refs/heads/bogus:refs/heads/bogus &&
+ git push . $commit:refs/heads/bogus &&
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckobjects true &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+
+ # fetch.fsck.<msg-id> does not fall back on fsck.<msg-id>
+ git --git-dir=dst/.git config fsck.missingEmail warn &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" $refspec &&
+
+ # receive.fsck.<unknownmsg-id> warns
+ git --git-dir=dst/.git config \
+ fetch.fsck.whatEver error &&
+
+ git --git-dir=dst/.git config \
+ fetch.fsck.missingEmail warn &&
+ git --git-dir=dst/.git fetch "file://$(pwd)" $refspec >act 2>&1 &&
+ grep "missingEmail" act &&
+ test_i18ngrep "Skipping unknown msg id.*whatever" act &&
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckobjects true &&
+ git --git-dir=dst/.git config \
+ fetch.fsck.missingEmail ignore &&
+ git --git-dir=dst/.git fetch "file://$(pwd)" $refspec >act 2>&1 &&
+ ! grep "missingEmail" act
+'
+
test_expect_success \
'receive.fsck.unterminatedHeader=warn triggers error' '
rm -rf dst &&
@@ -166,4 +273,15 @@ test_expect_success \
grep "Cannot demote unterminatedheader" act
'
+test_expect_success \
+ 'fetch.fsck.unterminatedHeader=warn triggers error' '
+ rm -rf dst &&
+ git init dst &&
+ git --git-dir=dst/.git config fetch.fsckobjects true &&
+ git --git-dir=dst/.git config \
+ fetch.fsck.unterminatedheader warn &&
+ test_must_fail git --git-dir=dst/.git fetch "file://$(pwd)" HEAD &&
+ grep "Cannot demote unterminatedheader" act
+'
+
test_done