Merge branch 'jk/credentials'

* jk/credentials:
  t: add test harness for external credential helpers
  credentials: add "store" helper
  strbuf: add strbuf_add*_urlencode
  Makefile: unix sockets may not available on some platforms
  credentials: add "cache" helper
  docs: end-user documentation for the credential subsystem
  credential: make relevance of http path configurable
  credential: add credential.*.username
  credential: apply helper config
  http: use credential API to get passwords
  credential: add function for parsing url components
  introduce credentials API
  t5550: fix typo
  test-lib: add test_config_global variant

Conflicts:
	strbuf.c

1 files changed, 245 insertions, 0 deletions

diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt
new file mode 100644
index 0000000000..21ca6a2553
--- /dev/null
+++ b/Documentation/technical/api-credentials.txt
@@ -0,0 +1,245 @@
+credentials API
+===============
+
+The credentials API provides an abstracted way of gathering username and
+password credentials from the user (even though credentials in the wider
+world can take many forms, in this document the word "credential" always
+refers to a username and password pair).
+
+Data Structures
+---------------
+
+`struct credential`::
+
+	This struct represents a single username/password combination
+	along with any associated context. All string fields should be
+	heap-allocated (or NULL if they are not known or not applicable).
+	The meaning of the individual context fields is the same as
+	their counterparts in the helper protocol; see the section below
+	for a description of each field.
++
+The `helpers` member of the struct is a `string_list` of helpers.  Each
+string specifies an external helper which will be run, in order, to
+either acquire or store credentials. See the section on credential
+helpers below.
++
+This struct should always be initialized with `CREDENTIAL_INIT` or
+`credential_init`.
+
+
+Functions
+---------
+
+`credential_init`::
+
+	Initialize a credential structure, setting all fields to empty.
+
+`credential_clear`::
+
+	Free any resources associated with the credential structure,
+	returning it to a pristine initialized state.
+
+`credential_fill`::
+
+	Instruct the credential subsystem to fill the username and
+	password fields of the passed credential struct by first
+	consulting helpers, then asking the user. After this function
+	returns, the username and password fields of the credential are
+	guaranteed to be non-NULL. If an error occurs, the function will
+	die().
+
+`credential_reject`::
+
+	Inform the credential subsystem that the provided credentials
+	have been rejected. This will cause the credential subsystem to
+	notify any helpers of the rejection (which allows them, for
+	example, to purge the invalid credentials from storage).  It
+	will also free() the username and password fields of the
+	credential and set them to NULL (readying the credential for
+	another call to `credential_fill`). Any errors from helpers are
+	ignored.
+
+`credential_approve`::
+
+	Inform the credential subsystem that the provided credentials
+	were successfully used for authentication.  This will cause the
+	credential subsystem to notify any helpers of the approval, so
+	that they may store the result to be used again.  Any errors
+	from helpers are ignored.
+
+`credential_from_url`::
+
+	Parse a URL into broken-down credential fields.
+
+Example
+-------
+
+The example below shows how the functions of the credential API could be
+used to login to a fictitious "foo" service on a remote host:
+
+-----------------------------------------------------------------------
+int foo_login(struct foo_connection *f)
+{
+	int status;
+	/*
+	 * Create a credential with some context; we don't yet know the
+	 * username or password.
+	 */
+
+	struct credential c = CREDENTIAL_INIT;
+	c.protocol = xstrdup("foo");
+	c.host = xstrdup(f->hostname);
+
+	/*
+	 * Fill in the username and password fields by contacting
+	 * helpers and/or asking the user. The function will die if it
+	 * fails.
+	 */
+	credential_fill(&c);
+
+	/*
+	 * Otherwise, we have a username and password. Try to use it.
+	 */
+	status = send_foo_login(f, c.username, c.password);
+	switch (status) {
+	case FOO_OK:
+		/* It worked. Store the credential for later use. */
+		credential_accept(&c);
+		break;
+	case FOO_BAD_LOGIN:
+		/* Erase the credential from storage so we don't try it
+		 * again. */
+		credential_reject(&c);
+		break;
+	default:
+		/*
+		 * Some other error occured. We don't know if the
+		 * credential is good or bad, so report nothing to the
+		 * credential subsystem.
+		 */
+	}
+
+	/* Free any associated resources. */
+	credential_clear(&c);
+
+	return status;
+}
+-----------------------------------------------------------------------
+
+
+Credential Helpers
+------------------
+
+Credential helpers are programs executed by git to fetch or save
+credentials from and to long-term storage (where "long-term" is simply
+longer than a single git process; e.g., credentials may be stored
+in-memory for a few minutes, or indefinitely on disk).
+
+Each helper is specified by a single string. The string is transformed
+by git into a command to be executed using these rules:
+
+  1. If the helper string begins with "!", it is considered a shell
+     snippet, and everything after the "!" becomes the command.
+
+  2. Otherwise, if the helper string begins with an absolute path, the
+     verbatim helper string becomes the command.
+
+  3. Otherwise, the string "git credential-" is prepended to the helper
+     string, and the result becomes the command.
+
+The resulting command then has an "operation" argument appended to it
+(see below for details), and the result is executed by the shell.
+
+Here are some example specifications:
+
+----------------------------------------------------
+# run "git credential-foo"
+foo
+
+# same as above, but pass an argument to the helper
+foo --bar=baz
+
+# the arguments are parsed by the shell, so use shell
+# quoting if necessary
+foo --bar="whitespace arg"
+
+# you can also use an absolute path, which will not use the git wrapper
+/path/to/my/helper --with-arguments
+
+# or you can specify your own shell snippet
+!f() { echo "password=`cat $HOME/.secret`"; }; f
+----------------------------------------------------
+
+Generally speaking, rule (3) above is the simplest for users to specify.
+Authors of credential helpers should make an effort to assist their
+users by naming their program "git-credential-$NAME", and putting it in
+the $PATH or $GIT_EXEC_PATH during installation, which will allow a user
+to enable it with `git config credential.helper $NAME`.
+
+When a helper is executed, it will have one "operation" argument
+appended to its command line, which is one of:
+
+`get`::
+
+	Return a matching credential, if any exists.
+
+`store`::
+
+	Store the credential, if applicable to the helper.
+
+`erase`::
+
+	Remove a matching credential, if any, from the helper's storage.
+
+The details of the credential will be provided on the helper's stdin
+stream. The credential is split into a set of named attributes.
+Attributes are provided to the helper, one per line. Each attribute is
+specified by a key-value pair, separated by an `=` (equals) sign,
+followed by a newline. The key may contain any bytes except `=`,
+newline, or NUL. The value may contain any bytes except newline or NUL.
+In both cases, all bytes are treated as-is (i.e., there is no quoting,
+and one cannot transmit a value with newline or NUL in it). The list of
+attributes is terminated by a blank line or end-of-file.
+
+Git will send the following attributes (but may not send all of
+them for a given credential; for example, a `host` attribute makes no
+sense when dealing with a non-network protocol):
+
+`protocol`::
+
+	The protocol over which the credential will be used (e.g.,
+	`https`).
+
+`host`::
+
+	The remote hostname for a network credential.
+
+`path`::
+
+	The path with which the credential will be used. E.g., for
+	accessing a remote https repository, this will be the
+	repository's path on the server.
+
+`username`::
+
+	The credential's username, if we already have one (e.g., from a
+	URL, from the user, or from a previously run helper).
+
+`password`::
+
+	The credential's password, if we are asking it to be stored.
+
+For a `get` operation, the helper should produce a list of attributes
+on stdout in the same format. A helper is free to produce a subset, or
+even no values at all if it has nothing useful to provide. Any provided
+attributes will overwrite those already known about by git.
+
+For a `store` or `erase` operation, the helper's output is ignored.
+If it fails to perform the requested operation, it may complain to
+stderr to inform the user. If it does not support the requested
+operation (e.g., a read-only store), it should silently ignore the
+request.
+
+If a helper receives any other operation, it should silently ignore the
+request. This leaves room for future operations to be added (older
+helpers will just ignore the new requests).