Sane use of test_expect_failure

Originally, test_expect_failure was designed to be the opposite
of test_expect_success, but this was a bad decision.  Most tests
run a series of commands that leads to the single command that
needs to be tested, like this:

    test_expect_{success,failure} 'test title' '
	setup1 &&
        setup2 &&
        setup3 &&
        what is to be tested
    '

And expecting a failure exit from the whole sequence misses the
point of writing tests.  Your setup$N that are supposed to
succeed may have failed without even reaching what you are
trying to test.  The only valid use of test_expect_failure is to
check a trivial single command that is expected to fail, which
is a minority in tests of Porcelain-ish commands.

This large-ish patch rewrites all uses of test_expect_failure to
use test_expect_success and rewrites the condition of what is
tested, like this:

    test_expect_success 'test title' '
	setup1 &&
        setup2 &&
        setup3 &&
        ! this command should fail
    '

test_expect_failure is redefined to serve as a reminder that
that test *should* succeed but due to a known breakage in git it
currently does not pass.  So if git-foo command should create a
file 'bar' but you discovered a bug that it doesn't, you can
write a test like this:

    test_expect_failure 'git-foo should create bar' '
        rm -f bar &&
        git foo &&
        test -f bar
    '

This construct acts similar to test_expect_success, but instead
of reporting "ok/FAIL" like test_expect_success does, the
outcome is reported as "FIXED/still broken".

Signed-off-by: Junio C Hamano <gitster@pobox.com>

1 files changed, 36 insertions, 32 deletions

diff --git a/t/t4103-apply-binary.sh b/t/t4103-apply-binary.sh
index 74f06ec730..7c25634fc2 100755
--- a/t/t4103-apply-binary.sh
+++ b/t/t4103-apply-binary.sh
@@ -46,21 +46,25 @@ test_expect_success 'stat binary diff (copy) -- should not fail.' \
 	'git-checkout master
 	 git apply --stat --summary C.diff'
 
-test_expect_failure 'check binary diff -- should fail.' \
-	'git-checkout master
-	 git apply --check B.diff'
-
-test_expect_failure 'check binary diff (copy) -- should fail.' \
-	'git-checkout master
-	 git apply --check C.diff'
-
-test_expect_failure 'check incomplete binary diff with replacement -- should fail.' \
-	'git-checkout master
-	 git apply --check --allow-binary-replacement B.diff'
+test_expect_success 'check binary diff -- should fail.' \
+	'git-checkout master &&
+	 ! git apply --check B.diff'
+
+test_expect_success 'check binary diff (copy) -- should fail.' \
+	'git-checkout master &&
+	 ! git apply --check C.diff'
+
+test_expect_success \
+	'check incomplete binary diff with replacement -- should fail.' '
+	git-checkout master &&
+	! git apply --check --allow-binary-replacement B.diff
+'
 
-test_expect_failure 'check incomplete binary diff with replacement (copy) -- should fail.' \
-	'git-checkout master
-	 git apply --check --allow-binary-replacement C.diff'
+test_expect_success \
+    'check incomplete binary diff with replacement (copy) -- should fail.' '
+	 git-checkout master &&
+	 ! git apply --check --allow-binary-replacement C.diff
+'
 
 test_expect_success 'check binary diff with replacement.' \
 	'git-checkout master
@@ -73,42 +77,42 @@ test_expect_success 'check binary diff with replacement (copy).' \
 # Now we start applying them.
 
 do_reset () {
-	rm -f file?
-	git-reset --hard
+	rm -f file? &&
+	git-reset --hard &&
 	git-checkout -f master
 }
 
-test_expect_failure 'apply binary diff -- should fail.' \
-	'do_reset
-	 git apply B.diff'
+test_expect_success 'apply binary diff -- should fail.' \
+	'do_reset &&
+	 ! git apply B.diff'
 
-test_expect_failure 'apply binary diff -- should fail.' \
-	'do_reset
-	 git apply --index B.diff'
+test_expect_success 'apply binary diff -- should fail.' \
+	'do_reset &&
+	 ! git apply --index B.diff'
 
-test_expect_failure 'apply binary diff (copy) -- should fail.' \
-	'do_reset
-	 git apply C.diff'
+test_expect_success 'apply binary diff (copy) -- should fail.' \
+	'do_reset &&
+	 ! git apply C.diff'
 
-test_expect_failure 'apply binary diff (copy) -- should fail.' \
-	'do_reset
-	 git apply --index C.diff'
+test_expect_success 'apply binary diff (copy) -- should fail.' \
+	'do_reset &&
+	 ! git apply --index C.diff'
 
 test_expect_success 'apply binary diff without replacement.' \
-	'do_reset
+	'do_reset &&
 	 git apply BF.diff'
 
 test_expect_success 'apply binary diff without replacement (copy).' \
-	'do_reset
+	'do_reset &&
 	 git apply CF.diff'
 
 test_expect_success 'apply binary diff.' \
-	'do_reset
+	'do_reset &&
 	 git apply --allow-binary-replacement --index BF.diff &&
 	 test -z "$(git diff --name-status binary)"'
 
 test_expect_success 'apply binary diff (copy).' \
-	'do_reset
+	'do_reset &&
 	 git apply --allow-binary-replacement --index CF.diff &&
 	 test -z "$(git diff --name-status binary)"'