Introduce a performance testing framework

This introduces a performance testing framework under t/perf/.  It
tries to be as close to the test-lib.sh infrastructure as possible,
and thus should be easy to get used to for git developers.

The following points were considered for the implementation:

1. You usually want to compare arbitrary revisions/build trees against
   each other.  They may not have the performance test under
   consideration, or even the perf-lib.sh infrastructure.

   To cope with this, the 'run' script lets you specify arbitrary
   build dirs and revisions.  It even automatically builds the revisions
   if it doesn't have them at hand yet.

2. Usually you would not want to run all tests.  It would take too
   long anyway.  The 'run' script lets you specify which tests to run;
   or you can also do it manually.  There is a Makefile for
   discoverability and 'make clean', but it is not meant for
   real-world use.

3. Creating test repos from scratch in every test is extremely
   time-consuming, and shipping or downloading such large/weird repos
   is out of the question.

   We leave this decision to the user.  Two different sizes of test
   repos can be configured, and the scripts just copy one or more of
   those (using hardlinks for the object store).  By default it tries
   to use the build tree's git.git repository.

   This is fairly fast and versatile.  Using a copy instead of a clone
   preserves many properties that the user may want to test for, such
   as lots of loose objects, unpacked refs, etc.

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

1 files changed, 82 insertions, 0 deletions

diff --git a/t/perf/run b/t/perf/run
new file mode 100755
index 0000000000..cfd70129bb
--- /dev/null
+++ b/t/perf/run
@@ -0,0 +1,82 @@
+#!/bin/sh
+
+case "$1" in
+	--help)
+		echo "usage: $0 [other_git_tree...] [--] [test_scripts]"
+		exit 0
+		;;
+esac
+
+die () {
+	echo >&2 "error: $*"
+	exit 1
+}
+
+run_one_dir () {
+	if test $# -eq 0; then
+		set -- p????-*.sh
+	fi
+	echo "=== Running $# tests in ${GIT_TEST_INSTALLED:-this tree} ==="
+	for t in "$@"; do
+		./$t $GIT_TEST_OPTS
+	done
+}
+
+unpack_git_rev () {
+	rev=$1
+	mkdir -p build/$rev
+	(cd "$(git rev-parse --show-cdup)" && git archive --format=tar $rev) |
+	(cd build/$rev && tar x)
+}
+build_git_rev () {
+	rev=$1
+	cp ../../config.mak build/$rev/config.mak
+	(cd build/$rev && make $GIT_PERF_MAKE_OPTS) ||
+	die "failed to build revision '$mydir'"
+}
+
+run_dirs_helper () {
+	mydir=${1%/}
+	shift
+	while test $# -gt 0 -a "$1" != -- -a ! -f "$1"; do
+		shift
+	done
+	if test $# -gt 0 -a "$1" = --; then
+		shift
+	fi
+	if [ ! -d "$mydir" ]; then
+		rev=$(git rev-parse --verify "$mydir" 2>/dev/null) ||
+		die "'$mydir' is neither a directory nor a valid revision"
+		if [ ! -d build/$rev ]; then
+			unpack_git_rev $rev
+		fi
+		build_git_rev $rev
+		mydir=build/$rev
+	fi
+	if test "$mydir" = .; then
+		unset GIT_TEST_INSTALLED
+	else
+		GIT_TEST_INSTALLED="$mydir/bin-wrappers"
+		export GIT_TEST_INSTALLED
+	fi
+	run_one_dir "$@"
+}
+
+run_dirs () {
+	while test $# -gt 0 -a "$1" != -- -a ! -f "$1"; do
+		run_dirs_helper "$@"
+		shift
+	done
+}
+
+GIT_PERF_AGGREGATING_LATER=t
+export GIT_PERF_AGGREGATING_LATER
+
+cd "$(dirname $0)"
+. ../../GIT-BUILD-OPTIONS
+
+if test $# = 0 -o "$1" = -- -o -f "$1"; then
+	set -- . "$@"
+fi
+run_dirs "$@"
+./aggregate.perl "$@"