From 93a5724613861e6cd85964c85f2fa0891caab258 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Fri, 6 Aug 2010 21:19:23 +0000
Subject: test-lib: Add support for multiple test prerequisites
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Change the test_have_prereq function in test-lib.sh to support a
comma-separated list of prerequisites. This is useful for tests that
need e.g. both POSIXPERM and SANITY.

The implementation was stolen from Junio C Hamano and Johannes Sixt,
the tests and documentation were not. See the "Tests in Cygwin" thread
in May 2009 for the originals:

    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 6 ++++++
 1 file changed, 6 insertions(+)

(limited to 't/README')

diff --git a/t/README b/t/README
index 0d1183c3e6..d07b67af6c 100644
--- a/t/README
+++ b/t/README
@@ -350,6 +350,12 @@ library for your script to use.
 	test_expect_success TTY 'git --paginate rev-list uses a pager' \
 	    ' ... '
 
+   You can also supply a comma-separated list of prerequisites, in the
+   rare case where your test depends on more than one:
+
+	test_expect_success PERL,PYTHON 'yo dawg' \
+	    ' test $(perl -E 'print eval "1 +" . qx[python -c "print 2"]') == "4" '
+
  - test_expect_failure [<prereq>] <message> <script>
 
    This is NOT the opposite of test_expect_success, but is used
-- 
cgit v1.2.3


From be53deef0d94b80b0a2df465d16f21ed29c3165b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Fri, 6 Aug 2010 21:19:25 +0000
Subject: t/README: Document the predefined test prerequisites
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The README for the test library suggested that you grep the
test-lib.sh for test_set_prereq to see what the preset prerequisites
were.

Remove that bit, and write a section explaining all the preset
prerequisites. Most of the text was lifted from from Junio C Hamano
and Johannes Sixt, See the "Tests in Cygwin" thread in May 2009 for
the originals:

    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 45 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 40 insertions(+), 5 deletions(-)

(limited to 't/README')

diff --git a/t/README b/t/README
index d07b67af6c..dc07939375 100644
--- a/t/README
+++ b/t/README
@@ -410,11 +410,12 @@ library for your script to use.
  - test_set_prereq SOME_PREREQ
 
    Set a test prerequisite to be used later with test_have_prereq. The
-   test-lib will set some prerequisites for you, e.g. PERL and PYTHON
-   which are derived from ./GIT-BUILD-OPTIONS (grep test_set_prereq
-   test-lib.sh for more). Others you can set yourself and use later
-   with either test_have_prereq directly, or the three argument
-   invocation of test_expect_success and test_expect_failure.
+   test-lib will set some prerequisites for you, see the
+   "Prerequisites" section below for a full list of these.
+
+   Others you can set yourself and use later with either
+   test_have_prereq directly, or the three argument invocation of
+   test_expect_success and test_expect_failure.
 
  - test_have_prereq SOME PREREQ
 
@@ -487,6 +488,40 @@ library for your script to use.
 		...
 	'
 
+Prerequisites
+-------------
+
+These are the prerequisites that the test library predefines with
+test_have_prereq.
+
+See the prereq argument to the test_* functions in the "Test harness
+library" section above and the "test_have_prereq" function for how to
+use these, and "test_set_prereq" for how to define your own.
+
+ - PERL & PYTHON
+
+   Git wasn't compiled with NO_PERL=YesPlease or
+   NO_PYTHON=YesPlease. Wrap any tests that need Perl or Python in
+   these.
+
+ - POSIXPERM
+
+   The filesystem supports POSIX style permission bits.
+
+ - BSLASHPSPEC
+
+   Backslashes in pathspec are not directory separators. This is not
+   set on Windows. See 6fd1106a for details.
+
+ - EXECKEEPSPID
+
+   The process retains the same pid across exec(2). See fb9a2bea for
+   details.
+
+ - SYMLINKS
+
+   The filesystem we're on supports symbolic links. E.g. a FAT
+   filesystem doesn't support these. See 704a3143 for details.
 
 Tips for Writing Tests
 ----------------------
-- 
cgit v1.2.3


From c91cfd19160be63e97f28dce8c9af421b61c9938 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Fri, 6 Aug 2010 22:09:09 +0000
Subject: tests: A SANITY test prereq for testing if we're root
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Some tests depend on not being able to write to files after chmod
-w. This doesn't work when running the tests as root.

Change test-lib.sh to test if this works, and if so it sets a new
SANITY test prerequisite. The tests that use this previously failed
when run under root.

There was already a test for this in t3600-rm.sh, added by Junio C
Hamano in 2283645 in 2006. That check now uses the new SANITY
prerequisite.

Some of this was resurrected from the "Tests in Cygwin" thread in May
2009:

    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 5 +++++
 1 file changed, 5 insertions(+)

(limited to 't/README')

diff --git a/t/README b/t/README
index dc07939375..28effb4437 100644
--- a/t/README
+++ b/t/README
@@ -523,6 +523,11 @@ use these, and "test_set_prereq" for how to define your own.
    The filesystem we're on supports symbolic links. E.g. a FAT
    filesystem doesn't support these. See 704a3143 for details.
 
+ - SANITY
+
+   Test is not run by root user, and an attempt to write to an
+   unwritable file is expected to fail correctly.
+
 Tips for Writing Tests
 ----------------------
 
-- 
cgit v1.2.3


From d15e9ebc5c8a627844275e02a2b15a101ce46d4a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Sun, 8 Aug 2010 14:49:25 +0000
Subject: t/README: Document the Smoke testing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Git now has a smoke testing service at http://smoke.git.nix.is that
anyone can send reports to. Change the t/README file to mention this.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

(limited to 't/README')

diff --git a/t/README b/t/README
index 28effb4437..5ed06f62d5 100644
--- a/t/README
+++ b/t/README
@@ -554,3 +554,69 @@ the purpose of t0000-basic.sh, which is to isolate that level of
 validation in one place.  Your test also ends up needing
 updating when such a change to the internal happens, so do _not_
 do it and leave the low level of validation to t0000-basic.sh.
+
+Smoke testing
+-------------
+
+The Git test suite has support for smoke testing. Smoke testing is
+when you submit the results of a test run to a central server for
+analysis and aggregation.
+
+Running a smoke tester is an easy and valuable way of contributing to
+Git development, particularly if you have access to an uncommon OS on
+obscure hardware.
+
+After building Git you can generate a smoke report like this in the
+"t" directory:
+
+    make clean smoke
+
+You can also pass arguments via the environment. This should make it
+faster:
+
+    GIT_TEST_OPTS='--root=/dev/shm' TEST_JOBS=10 make clean smoke
+
+The "smoke" target will run the Git test suite with Perl's
+"TAP::Harness" module, and package up the results in a .tar.gz archive
+with "TAP::Harness::Archive". The former is included with Perl v5.10.1
+or later, but you'll need to install the latter from the CPAN. See the
+"Test coverage" section above for how you might do that.
+
+Once the "smoke" target finishes you'll see a message like this:
+
+    TAP Archive created at <path to git>/t/test-results/git-smoke.tar.gz
+
+To upload the smoke report you need to have curl(1) installed, then
+do:
+
+    make smoke_report
+
+To upload the report anonymously. Hopefully that'll return something
+like "Reported #7 added.".
+
+If you're going to be uploading reports frequently please request a
+user account by E-Mailing gitsmoke@v.nix.is. Once you have a username
+and password you'll be able to do:
+
+    SMOKE_USERNAME=<username> SMOKE_PASSWORD=<password> make smoke_report
+
+Once the report is uploaded it'll be made available at
+http://smoke.git.nix.is, here's an overview of Recent Smoke Reports
+for Git:
+
+    http://smoke.git.nix.is/app/projects/smoke_reports/1
+
+The reports will also be mirrored to GitHub every few hours:
+
+    http://github.com/gitsmoke/smoke-reports
+
+The Smolder SQLite database is also mirrored and made available for
+download:
+
+    http://github.com/gitsmoke/smoke-database
+
+Note that the database includes hashed (with crypt()) user passwords
+and E-Mail addresses. Don't use a valuable password for the smoke
+service if you have an account, or an E-Mail address you don't want to
+be publicly known. The user accounts are just meant to be convenient
+labels, they're not meant to be secure.
-- 
cgit v1.2.3


From e38efac87d4542d2368ad772d1e9dd51ab650acb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Sun, 8 Aug 2010 14:49:27 +0000
Subject: t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The smoke server supports a free form text field with comments about a
report, and a comma delimited list of tags. Change the smoke_report
target to expose this functionality. Now smokers can send more data
that explains and categorizes the reports they're submitting.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 7 +++++++
 1 file changed, 7 insertions(+)

(limited to 't/README')

diff --git a/t/README b/t/README
index 5ed06f62d5..336871fe42 100644
--- a/t/README
+++ b/t/README
@@ -600,6 +600,13 @@ and password you'll be able to do:
 
     SMOKE_USERNAME=<username> SMOKE_PASSWORD=<password> make smoke_report
 
+You can also add an additional comment to attach to the report, and/or
+a comma separated list of tags:
+
+    SMOKE_USERNAME=<username> SMOKE_PASSWORD=<password> \
+        SMOKE_COMMENT=<comment> SMOKE_TAGS=<tags> \
+        make smoke_report
+
 Once the report is uploaded it'll be made available at
 http://smoke.git.nix.is, here's an overview of Recent Smoke Reports
 for Git:
-- 
cgit v1.2.3


From 0c357544b0d4415c464fb8b35593b21db1d1f7f7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Sun, 25 Jul 2010 19:52:44 +0000
Subject: t/README: A new section about test coverage
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Document how test writers can generate coverage reports, to ensure
that their tests are really testing the code they think they're
testing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

(limited to 't/README')

diff --git a/t/README b/t/README
index 336871fe42..468876de0d 100644
--- a/t/README
+++ b/t/README
@@ -268,6 +268,9 @@ Do:
 	git push gh &&
 	test ...
 
+ - Check the test coverage for your tests. See the "Test coverage"
+   below.
+
 Don't:
 
  - exit() within a <script> part.
@@ -555,6 +558,45 @@ validation in one place.  Your test also ends up needing
 updating when such a change to the internal happens, so do _not_
 do it and leave the low level of validation to t0000-basic.sh.
 
+Test coverage
+-------------
+
+You can use the coverage tests to find code paths that are not being
+used or properly exercised yet.
+
+To do that, run the coverage target at the top-level (not in the t/
+directory):
+
+    make coverage
+
+That'll compile Git with GCC's coverage arguments, and generate a test
+report with gcov after the tests finish. Running the coverage tests
+can take a while, since running the tests in parallel is incompatible
+with GCC's coverage mode.
+
+After the tests have run you can generate a list of untested
+functions:
+
+    make coverage-untested-functions
+
+You can also generate a detailed per-file HTML report using the
+Devel::Cover module. To install it do:
+
+   # On Debian or Ubuntu:
+   sudo aptitude install libdevel-cover-perl
+
+   # From the CPAN with cpanminus
+   curl -L http://cpanmin.us | perl - --sudo --self-upgrade
+   cpanm --sudo Devel::Cover
+
+Then, at the top-level:
+
+    make cover_db_html
+
+That'll generate a detailed cover report in the "cover_db_html"
+directory, which you can then copy to a webserver, or inspect locally
+in a browser.
+
 Smoke testing
 -------------
 
-- 
cgit v1.2.3


From e8b55f5c58d1d9e5d5cd5b24d96b7d3b51047440 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Sun, 25 Jul 2010 19:52:45 +0000
Subject: t/README: Add a note about the dangers of coverage chasing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Having no coverage at all is almost always a bad sign, but trying to
attain 100% coverage everywhere is usually a waste of time. Add a
paragraph to explain this to future test writers.

Inspired-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 9 +++++++++
 1 file changed, 9 insertions(+)

(limited to 't/README')

diff --git a/t/README b/t/README
index 468876de0d..e49534a1bf 100644
--- a/t/README
+++ b/t/README
@@ -271,6 +271,15 @@ Do:
  - Check the test coverage for your tests. See the "Test coverage"
    below.
 
+   Don't blindly follow test coverage metrics, they're a good way to
+   spot if you've missed something. If a new function you added
+   doesn't have any coverage you're probably doing something wrong,
+   but having 100% coverage doesn't necessarily mean that you tested
+   everything.
+
+   Tests that are likely to smoke out future regressions are better
+   than tests that just inflate the coverage metrics.
+
 Don't:
 
  - exit() within a <script> part.
-- 
cgit v1.2.3


From 99d9050d256d1a928ce6d13fe275d8fd4bfcbdee Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>
Date: Wed, 28 Jul 2010 10:34:59 +0000
Subject: t/README: Update "Skipping tests" to align with best practices
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The example I initially added to "Skipping tests" wasn't very
good. We'd rather skip tests using the three-arg prereq form to the
test_* functions, not bail out with a skip message.

Change the documentation to reflect that, but retain the bailout
example under a disclaimer which explains that it's probably not a
good idea to use it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/README | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

(limited to 't/README')

diff --git a/t/README b/t/README
index e49534a1bf..118e50df72 100644
--- a/t/README
+++ b/t/README
@@ -319,9 +319,21 @@ Keep in mind:
 Skipping tests
 --------------
 
-If you need to skip all the remaining tests you should set skip_all
-and immediately call test_done. The string you give to skip_all will
-be used as an explanation for why the test was skipped. for instance:
+If you need to skip tests you should do so be using the three-arg form
+of the test_* functions (see the "Test harness library" section
+below), e.g.:
+
+    test_expect_success PERL 'I need Perl' "
+        '$PERL_PATH' -e 'hlagh() if unf_unf()'
+    "
+
+The advantage of skipping tests like this is that platforms that don't
+have the PERL and other optional dependencies get an indication of how
+many tests they're missing.
+
+If the test code is too hairy for that (i.e. does a lot of setup work
+outside test assertions) you can also skip all remaining tests by
+setting skip_all and immediately call test_done:
 
 	if ! test_have_prereq PERL
 	then
@@ -329,6 +341,9 @@ be used as an explanation for why the test was skipped. for instance:
 	    test_done
 	fi
 
+The string you give to skip_all will be used as an explanation for why
+the test was skipped.
+
 End with test_done
 ------------------
 
-- 
cgit v1.2.3