diff options
Diffstat (limited to 'contrib')
94 files changed, 8262 insertions, 6397 deletions
diff --git a/contrib/buildsystems/engine.pl b/contrib/buildsystems/engine.pl index 23da787dc5..23da787dc5 100644..100755 --- a/contrib/buildsystems/engine.pl +++ b/contrib/buildsystems/engine.pl diff --git a/contrib/buildsystems/generate b/contrib/buildsystems/generate index bc10f25ff2..bc10f25ff2 100644..100755 --- a/contrib/buildsystems/generate +++ b/contrib/buildsystems/generate diff --git a/contrib/buildsystems/parse.pl b/contrib/buildsystems/parse.pl index c9656ece99..c9656ece99 100644..100755 --- a/contrib/buildsystems/parse.pl +++ b/contrib/buildsystems/parse.pl diff --git a/contrib/ciabot/INSTALL b/contrib/ciabot/INSTALL deleted file mode 100644 index 7222961d35..0000000000 --- a/contrib/ciabot/INSTALL +++ /dev/null @@ -1,54 +0,0 @@ -= Installation instructions = - -Two scripts are included. The Python one (ciabot.py) is faster and -more capable; the shell one (ciabot.sh) is a fallback in case Python -gives your git hosting site indigestion. (I know of no such sites.) - -It is no longer necessary to modify the script in order to put it -in place; in fact, this is now discouraged. It is entirely -configurable with the following git config variables: - -ciabot.project = name of the project -ciabot.repo = name of the project repo for gitweb/cgit purposes -ciabot.xmlrpc = if true, ship notifications via XML-RPC -ciabot.revformat = format in which the revision is shown - -The revformat variable may have the following values -raw -> full hex ID of commit -short -> first 12 chars of hex ID -describe -> describe relative to last tag, falling back to short - -ciabot.project defaults to the directory name of the repository toplevel. -ciabot.repo defaults to ciabot.project lowercased. -ciabot.xmlrpc defaults to True -ciabot.revformat defaults to 'describe'. - -This means that in the normal case you need not do any configuration at all, -however setting ciabot.project will allow the hook to run slightly faster. - -Once you've set these variables, try your script with -n to see the -notification message dumped to stdout and verify that it looks sane. - -To live-test these scripts, your project needs to have been registered with -the CIA site. Here are the steps: - -1. Open an IRC window on irc://freenode/commits or your registered - project IRC channel. - -2. Run ciabot.py and/or ciabot.sh from any directory under git - control. - -You should see a notification on the channel for your most recent commit. - -After verifying correct function, install one of these scripts either -in a post-commit hook or in an update hook. - -In post-commit, run it without arguments. It will query for -current HEAD and the latest commit ID to get the information it -needs. - -In update, call it with a refname followed by a list of commits: -You want to reverse the order git rev-list emits because it lists -from most recent to oldest. - -/path/to/ciabot.py ${refname} $(git rev-list ${oldhead}..${newhead} | tac) diff --git a/contrib/ciabot/README b/contrib/ciabot/README deleted file mode 100644 index 2dfe1f91f5..0000000000 --- a/contrib/ciabot/README +++ /dev/null @@ -1,11 +0,0 @@ -These are hook scripts for the CIA notification service at <http://cia.vc/> - -They are maintained by Eric S. Raymond <esr@thyrsus.com>. There is an -upstream resource page for them at <http://www.catb.org/esr/ciabot/>, -but they are unlikely to change rapidly. - -You probably want the Python version; it's faster, more capable, and -better documented. The shell version is maintained only as a fallback -for use on hosting sites that don't permit Python hook scripts. - -See the file INSTALL for installation instructions. diff --git a/contrib/ciabot/ciabot.py b/contrib/ciabot/ciabot.py deleted file mode 100755 index 36b5665ff8..0000000000 --- a/contrib/ciabot/ciabot.py +++ /dev/null @@ -1,255 +0,0 @@ -#!/usr/bin/env python -# Copyright (c) 2010 Eric S. Raymond <esr@thyrsus.com> -# Distributed under BSD terms. -# -# This script contains porcelain and porcelain byproducts. -# It's Python because the Python standard libraries avoid portability/security -# issues raised by callouts in the ancestral Perl and sh scripts. It should -# be compatible back to Python 2.1.5 -# -# usage: ciabot.py [-V] [-n] [-p projectname] [refname [commits...]] -# -# This script is meant to be run either in a post-commit hook or in an -# update hook. Try it with -n to see the notification mail dumped to -# stdout and verify that it looks sane. With -V it dumps its version -# and exits. -# -# In post-commit, run it without arguments. It will query for -# current HEAD and the latest commit ID to get the information it -# needs. -# -# In update, call it with a refname followed by a list of commits: -# You want to reverse the order git rev-list emits because it lists -# from most recent to oldest. -# -# /path/to/ciabot.py ${refname} $(git rev-list ${oldhead}..${newhead} | tac) -# -# Configuration variables affecting this script: -# -# ciabot.project = name of the project -# ciabot.repo = name of the project repo for gitweb/cgit purposes -# ciabot.xmlrpc = if true (default), ship notifications via XML-RPC -# ciabot.revformat = format in which the revision is shown -# -# ciabot.project defaults to the directory name of the repository toplevel. -# ciabot.repo defaults to ciabot.project lowercased. -# -# This means that in the normal case you need not do any configuration at all, -# but setting the project name will speed it up slightly. -# -# The revformat variable may have the following values -# raw -> full hex ID of commit -# short -> first 12 chars of hex ID -# describe = -> describe relative to last tag, falling back to short -# The default is 'describe'. -# -# Note: the CIA project now says only XML-RPC is reliable, so -# we default to that. -# - -import sys -if sys.hexversion < 0x02000000: - # The limiter is the xml.sax module - sys.stderr.write("ciabot.py: requires Python 2.0.0 or later.\n") - sys.exit(1) - -import os, commands, socket, urllib -from xml.sax.saxutils import escape - -# Changeset URL prefix for your repo: when the commit ID is appended -# to this, it should point at a CGI that will display the commit -# through gitweb or something similar. The defaults will probably -# work if you have a typical gitweb/cgit setup. -# -#urlprefix="http://%(host)s/cgi-bin/gitweb.cgi?p=%(repo)s;a=commit;h=" -urlprefix="http://%(host)s/cgi-bin/cgit.cgi/%(repo)s/commit/?id=" - -# The service used to turn your gitwebbish URL into a tinyurl so it -# will take up less space on the IRC notification line. -tinyifier = "http://tinyurl.com/api-create.php?url=" - -# The template used to generate the XML messages to CIA. You can make -# visible changes to the IRC-bot notification lines by hacking this. -# The default will produce a notfication line that looks like this: -# -# ${project}: ${author} ${repo}:${branch} * ${rev} ${files}: ${logmsg} ${url} -# -# By omitting $files you can collapse the files part to a single slash. -xml = '''\ -<message> - <generator> - <name>CIA Python client for Git</name> - <version>%(version)s</version> - <url>%(generator)s</url> - </generator> - <source> - <project>%(project)s</project> - <branch>%(repo)s:%(branch)s</branch> - </source> - <timestamp>%(ts)s</timestamp> - <body> - <commit> - <author>%(author)s</author> - <revision>%(rev)s</revision> - <files> - %(files)s - </files> - <log>%(logmsg)s %(url)s</log> - <url>%(url)s</url> - </commit> - </body> -</message> -''' - -# -# No user-serviceable parts below this line: -# - -# Where to ship e-mail notifications. -toaddr = "cia@cia.vc" - -# Identify the generator script. -# Should only change when the script itself gets a new home and maintainer. -generator = "http://www.catb.org/~esr/ciabot.py" -version = "3.6" - -def do(command): - return commands.getstatusoutput(command)[1] - -def report(refname, merged, xmlrpc=True): - "Generate a commit notification to be reported to CIA" - - # Try to tinyfy a reference to a web view for this commit. - try: - url = open(urllib.urlretrieve(tinyifier + urlprefix + merged)[0]).read() - except: - url = urlprefix + merged - - branch = os.path.basename(refname) - - # Compute a description for the revision - if revformat == 'raw': - rev = merged - elif revformat == 'short': - rev = '' - else: # revformat == 'describe' - rev = do("git describe %s 2>/dev/null" % merged) - if not rev: - rev = merged[:12] - - # Extract the meta-information for the commit - files=do("git diff-tree -r --name-only '"+ merged +"' | sed -e '1d' -e 's-.*-<file>&</file>-'") - metainfo = do("git log -1 '--pretty=format:%an <%ae>%n%at%n%s' " + merged) - (author, ts, logmsg) = metainfo.split("\n") - logmsg = escape(logmsg) - - # This discards the part of the author's address after @. - # Might be be nice to ship the full email address, if not - # for spammers' address harvesters - getting this wrong - # would make the freenode #commits channel into harvester heaven. - author = escape(author.replace("<", "").split("@")[0].split()[-1]) - - # This ignores the timezone. Not clear what to do with it... - ts = ts.strip().split()[0] - - context = locals() - context.update(globals()) - - out = xml % context - mail = '''\ -Message-ID: <%(merged)s.%(author)s@%(project)s> -From: %(fromaddr)s -To: %(toaddr)s -Content-type: text/xml -Subject: DeliverXML - -%(out)s''' % locals() - - if xmlrpc: - return out - else: - return mail - -if __name__ == "__main__": - import getopt - - # Get all config variables - revformat = do("git config --get ciabot.revformat") - project = do("git config --get ciabot.project") - repo = do("git config --get ciabot.repo") - xmlrpc = do("git config --get ciabot.xmlrpc") - xmlrpc = not (xmlrpc and xmlrpc == "false") - - host = socket.getfqdn() - fromaddr = "CIABOT-NOREPLY@" + host - - try: - (options, arguments) = getopt.getopt(sys.argv[1:], "np:xV") - except getopt.GetoptError, msg: - print "ciabot.py: " + str(msg) - raise SystemExit, 1 - - notify = True - for (switch, val) in options: - if switch == '-p': - project = val - elif switch == '-n': - notify = False - elif switch == '-x': - xmlrpc = True - elif switch == '-V': - print "ciabot.py: version", version - sys.exit(0) - - # The project variable defaults to the name of the repository toplevel. - if not project: - here = os.getcwd() - while True: - if os.path.exists(os.path.join(here, ".git")): - project = os.path.basename(here) - break - elif here == '/': - sys.stderr.write("ciabot.py: no .git below root!\n") - sys.exit(1) - here = os.path.dirname(here) - - if not repo: - repo = project.lower() - - urlprefix = urlprefix % globals() - - # The script wants a reference to head followed by the list of - # commit ID to report about. - if len(arguments) == 0: - refname = do("git symbolic-ref HEAD 2>/dev/null") - merges = [do("git rev-parse HEAD")] - else: - refname = arguments[0] - merges = arguments[1:] - - if notify: - if xmlrpc: - import xmlrpclib - server = xmlrpclib.Server('http://cia.vc/RPC2'); - else: - import smtplib - server = smtplib.SMTP('localhost') - - for merged in merges: - message = report(refname, merged, xmlrpc) - if not notify: - print message - elif xmlrpc: - try: - # RPC server is flaky, this can fail due to timeout. - server.hub.deliver(message) - except socket.error, e: - sys.stderr.write("%s\n" % e) - else: - server.sendmail(fromaddr, [toaddr], message) - - if notify: - if not xmlrpc: - server.quit() - -#End diff --git a/contrib/ciabot/ciabot.sh b/contrib/ciabot/ciabot.sh deleted file mode 100755 index 3fbbc534ae..0000000000 --- a/contrib/ciabot/ciabot.sh +++ /dev/null @@ -1,233 +0,0 @@ -#!/bin/sh -# Distributed under the terms of the GNU General Public License v2 -# Copyright (c) 2006 Fernando J. Pereda <ferdy@gentoo.org> -# Copyright (c) 2008 Natanael Copa <natanael.copa@gmail.com> -# Copyright (c) 2010 Eric S. Raymond <esr@thyrsus.com> -# Assistance and review by Petr Baudis, author of ciabot.pl, -# is gratefully acknowledged. -# -# This is a version 3.x of ciabot.sh; use -V to find the exact -# version. Versions 1 and 2 were shipped in 2006 and 2008 and are not -# version-stamped. The version 2 maintainer has passed the baton. -# -# Note: This script should be considered obsolete. -# There is a faster, better-documented rewrite in Python: find it as ciabot.py -# Use this only if your hosting site forbids Python hooks. -# It requires: git(1), hostname(1), cut(1), sendmail(1), and wget(1). -# -# Originally based on Git ciabot.pl by Petr Baudis. -# This script contains porcelain and porcelain byproducts. -# -# usage: ciabot.sh [-V] [-n] [-p projectname] [refname commit] -# -# This script is meant to be run either in a post-commit hook or in an -# update hook. Try it with -n to see the notification mail dumped to -# stdout and verify that it looks sane. With -V it dumps its version -# and exits. -# -# In post-commit, run it without arguments. It will query for -# current HEAD and the latest commit ID to get the information it -# needs. -# -# In update, you have to call it once per merged commit: -# -# refname=$1 -# oldhead=$2 -# newhead=$3 -# for merged in $(git rev-list ${oldhead}..${newhead} | tac) ; do -# /path/to/ciabot.sh ${refname} ${merged} -# done -# -# The reason for the tac call is that git rev-list emits commits from -# most recent to least - better to ship notifactions from oldest to newest. -# -# Configuration variables affecting this script: -# -# ciabot.project = name of the project -# ciabot.repo = name of the project repo for gitweb/cgit purposes -# ciabot.revformat = format in which the revision is shown -# -# ciabot.project defaults to the directory name of the repository toplevel. -# ciabot.repo defaults to ciabot.project lowercased. -# -# This means that in the normal case you need not do any configuration at all, -# but setting the project name will speed it up slightly. -# -# The revformat variable may have the following values -# raw -> full hex ID of commit -# short -> first 12 chars of hex ID -# describe = -> describe relative to last tag, falling back to short -# The default is 'describe'. -# -# Note: the shell ancestors of this script used mail, not XML-RPC, in -# order to avoid stalling until timeout when the CIA XML-RPC server is -# down. It is unknown whether this is still an issue in 2010, but -# XML-RPC would be annoying to do from sh in any case. (XML-RPC does -# have the advantage that it guarantees notification of multiple commits -# shpped from an update in their actual order.) -# - -# The project as known to CIA. You can set this with a -p option, -# or let it default to the directory name of the repo toplevel. -project=$(git config --get ciabot.project) - -if [ -z $project ] -then - here=`pwd`; - while :; do - if [ -d $here/.git ] - then - project=`basename $here` - break - elif [ $here = '/' ] - then - echo "ciabot.sh: no .git below root!" - exit 1 - fi - here=`dirname $here` - done -fi - -# Name of the repo for gitweb/cgit purposes -repo=$(git config --get ciabot.repo) -[ -z $repo] && repo=$(echo "${project}" | tr '[A-Z]' '[a-z]') - -# What revision format do we want in the summary? -revformat=$(git config --get ciabot.revformat) - -# Fully qualified domain name of the repo host. You can hardwire this -# to make the script faster. The -f option works under Linux and FreeBSD, -# but not OpenBSD and NetBSD. But under OpenBSD and NetBSD, -# hostname without options gives the FQDN. -if hostname -f >/dev/null 2>&1 -then - hostname=`hostname -f` -else - hostname=`hostname` -fi - -# Changeset URL prefix for your repo: when the commit ID is appended -# to this, it should point at a CGI that will display the commit -# through gitweb or something similar. The defaults will probably -# work if you have a typical gitweb/cgit setup. -#urlprefix="http://${host}/cgi-bin/gitweb.cgi?p=${repo};a=commit;h=" -urlprefix="http://${host}/cgi-bin/cgit.cgi/${repo}/commit/?id=" - -# -# You probably will not need to change the following: -# - -# Identify the script. The 'generator' variable should change only -# when the script itself gets a new home and maintainer. -generator="http://www.catb.org/~esr/ciabot/ciabot.sh" -version=3.5 - -# Addresses for the e-mail -from="CIABOT-NOREPLY@${hostname}" -to="cia@cia.vc" - -# SMTP client to use - may need to edit the absolute pathname for your system -sendmail="sendmail -t -f ${from}" - -# -# No user-serviceable parts below this line: -# - -# Should include all places sendmail is likely to lurk. -PATH="$PATH:/usr/sbin/" - -mode=mailit -while getopts pnV opt -do - case $opt in - p) project=$2; shift ; shift ;; - n) mode=dumpit; shift ;; - V) echo "ciabot.sh: version $version"; exit 0; shift ;; - esac -done - -# Cough and die if user has not specified a project -if [ -z "$project" ] -then - echo "ciabot.sh: no project specified, bailing out." >&2 - exit 1 -fi - -if [ $# -eq 0 ] ; then - refname=$(git symbolic-ref HEAD 2>/dev/null) - merged=$(git rev-parse HEAD) -else - refname=$1 - merged=$2 -fi - -# This tries to turn your gitwebbish URL into a tinyurl so it will take up -# less space on the IRC notification line. Some repo sites (I'm looking at -# you, berlios.de!) forbid wget calls for security reasons. On these, -# the code will fall back to the full un-tinyfied URL. -longurl=${urlprefix}${merged} -url=$(wget -O - -q http://tinyurl.com/api-create.php?url=${longurl} 2>/dev/null) -if [ -z "$url" ]; then - url="${longurl}" -fi - -refname=${refname##refs/heads/} - -case $revformat in -raw) rev=$merged ;; -short) rev='' ;; -*) rev=$(git describe ${merged} 2>/dev/null) ;; -esac -[ -z ${rev} ] && rev=$(echo "$merged" | cut -c 1-12) - -# We discard the part of the author's address after @. -# Might be nice to ship the full email address, if not -# for spammers' address harvesters - getting this wrong -# would make the freenode #commits channel into harvester heaven. -author=$(git log -1 '--pretty=format:%an <%ae>' $merged) -author=$(echo "$author" | sed -n -e '/^.*<\([^@]*\).*$/s--\1-p') - -logmessage=$(git log -1 '--pretty=format:%s' $merged) -ts=$(git log -1 '--pretty=format:%at' $merged) -files=$(git diff-tree -r --name-only ${merged} | sed -e '1d' -e 's-.*-<file>&</file>-') - -out=" -<message> - <generator> - <name>CIA Shell client for Git</name> - <version>${version}</version> - <url>${generator}</url> - </generator> - <source> - <project>${project}</project> - <branch>$repo:${refname}</branch> - </source> - <timestamp>${ts}</timestamp> - <body> - <commit> - <author>${author}</author> - <revision>${rev}</revision> - <files> - ${files} - </files> - <log>${logmessage} ${url}</log> - <url>${url}</url> - </commit> - </body> -</message>" - -if [ "$mode" = "dumpit" ] -then - sendmail=cat -fi - -${sendmail} << EOM -Message-ID: <${merged}.${author}@${project}> -From: ${from} -To: ${to} -Content-type: text/xml -Subject: DeliverXML -${out} -EOM - -# vim: set tw=70 : diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash index 6c3bafeea5..482ca84b45 100644 --- a/contrib/completion/git-completion.bash +++ b/contrib/completion/git-completion.bash @@ -1,5 +1,3 @@ -#!bash -# # bash/zsh completion support for core Git. # # Copyright (C) 2006,2007 Shawn O. Pearce <spearce@spearce.org> @@ -18,11 +16,17 @@ # # To use these routines: # -# 1) Copy this file to somewhere (e.g. ~/.git-completion.sh). +# 1) Copy this file to somewhere (e.g. ~/.git-completion.bash). # 2) Add the following line to your .bashrc/.zshrc: -# source ~/.git-completion.sh +# source ~/.git-completion.bash # 3) Consider changing your PS1 to also show the current branch, # see git-prompt.sh for details. +# +# If you use complex aliases of form '!f() { ... }; f', you can use the null +# command ':' as the first command in the function body to declare the desired +# completion style. For example '!f() { : git commit ; ... }; f' will +# tell the completion to use commit completion. This also works with aliases +# of form "!sh -c '...'". For example, "!sh -c ': git commit ; ... '". case "$COMP_WORDBREAKS" in *:*) : great ;; @@ -33,8 +37,6 @@ esac # returns location of .git repo __gitdir () { - # Note: this function is duplicated in git-prompt.sh - # When updating it, make sure you update the other one to match. if [ -z "${1-}" ]; then if [ -n "${__git_dir-}" ]; then echo "$__git_dir" @@ -182,9 +184,9 @@ _get_comp_words_by_ref () } fi -__gitcompadd () +__gitcompappend () { - local i=0 + local x i=${#COMPREPLY[@]} for x in $1; do if [[ "$x" == "$3"* ]]; then COMPREPLY[i++]="$2$x$4" @@ -192,6 +194,12 @@ __gitcompadd () done } +__gitcompadd () +{ + COMPREPLY=() + __gitcompappend "$@" +} + # Generates completion reply, appending a space to possible completion words, # if necessary. # It accepts 1 to 4 arguments: @@ -222,6 +230,14 @@ __gitcomp () esac } +# Variation of __gitcomp_nl () that appends to the existing list of +# completion candidates, COMPREPLY. +__gitcomp_nl_append () +{ + local IFS=$'\n' + __gitcompappend "$1" "${2-}" "${3-$cur}" "${4- }" +} + # Generates completion reply from newline-separated possible completion words # by appending a space to all of them. # It accepts 1 to 4 arguments: @@ -233,8 +249,8 @@ __gitcomp () # appended. __gitcomp_nl () { - local IFS=$'\n' - __gitcompadd "$1" "${2-}" "${3-$cur}" "${4- }" + COMPREPLY=() + __gitcomp_nl_append "$@" } # Generates completion reply with compgen from newline-separated possible @@ -265,16 +281,12 @@ __gitcomp_file () # argument, and using the options specified in the second argument. __git_ls_files_helper () { - ( - test -n "${CDPATH+set}" && unset CDPATH - cd "$1" - if [ "$2" == "--committable" ]; then - git diff-index --name-only --relative HEAD - else - # NOTE: $2 is not quoted in order to support multiple options - git ls-files --exclude-standard $2 - fi - ) 2>/dev/null + if [ "$2" == "--committable" ]; then + git -C "$1" diff-index --name-only --relative HEAD + else + # NOTE: $2 is not quoted in order to support multiple options + git -C "$1" ls-files --exclude-standard $2 + fi 2>/dev/null } @@ -372,7 +384,8 @@ __git_refs () ;; *) echo "HEAD" - git for-each-ref --format="%(refname:short)" -- "refs/remotes/$dir/" | sed -e "s#^$dir/##" + git for-each-ref --format="%(refname:short)" -- \ + "refs/remotes/$dir/" 2>/dev/null | sed -e "s#^$dir/##" ;; esac } @@ -398,12 +411,9 @@ __git_refs_remotes () __git_remotes () { - local i IFS=$'\n' d="$(__gitdir)" + local d="$(__gitdir)" test -d "$d/remotes" && ls -1 "$d/remotes" - for i in $(git --git-dir="$d" config --get-regexp 'remote\..*\.url' 2>/dev/null); do - i="${i#remote.}" - echo "${i/.url*/}" - done + git --git-dir="$d" remote } __git_list_merge_strategies () @@ -506,7 +516,7 @@ __git_complete_index_file () ;; esac - __gitcomp_file "$(__git_index_files "$1" "$pfx")" "$pfx" "$cur_" + __gitcomp_file "$(__git_index_files "$1" ${pfx:+"$pfx"})" "$pfx" "$cur_" } __git_complete_file () @@ -650,12 +660,13 @@ __git_list_porcelain_commands () cat-file) : plumbing;; check-attr) : plumbing;; check-ignore) : plumbing;; + check-mailmap) : plumbing;; check-ref-format) : plumbing;; checkout-index) : plumbing;; commit-tree) : plumbing;; count-objects) : infrequent;; - credential-cache) : credentials helper;; - credential-store) : credentials helper;; + credential) : credentials;; + credential-*) : credentials helper;; cvsexportcommit) : export;; cvsimport) : import;; cvsserver) : daemon;; @@ -674,7 +685,6 @@ __git_list_porcelain_commands () index-pack) : plumbing;; init-db) : deprecated;; local-fetch) : plumbing;; - lost-found) : infrequent;; ls-files) : plumbing;; ls-remote) : plumbing;; ls-tree) : plumbing;; @@ -688,14 +698,12 @@ __git_list_porcelain_commands () pack-refs) : plumbing;; parse-remote) : plumbing;; patch-id) : plumbing;; - peek-remote) : plumbing;; prune) : plumbing;; prune-packed) : plumbing;; quiltimport) : import;; read-tree) : plumbing;; receive-pack) : plumbing;; remote-*) : transport;; - repo-config) : deprecated;; rerere) : plumbing;; rev-list) : plumbing;; rev-parse) : plumbing;; @@ -708,7 +716,6 @@ __git_list_porcelain_commands () ssh-*) : transport;; stripspace) : plumbing;; symbolic-ref) : plumbing;; - tar-tree) : deprecated;; unpack-file) : plumbing;; unpack-objects) : plumbing;; update-index) : plumbing;; @@ -728,35 +735,28 @@ __git_list_porcelain_commands () __git_porcelain_commands= __git_compute_porcelain_commands () { - __git_compute_all_commands test -n "$__git_porcelain_commands" || __git_porcelain_commands=$(__git_list_porcelain_commands) } -__git_pretty_aliases () +# Lists all set config variables starting with the given section prefix, +# with the prefix removed. +__git_get_config_variables () { - local i IFS=$'\n' - for i in $(git --git-dir="$(__gitdir)" config --get-regexp "pretty\..*" 2>/dev/null); do - case "$i" in - pretty.*) - i="${i#pretty.}" - echo "${i/ */}" - ;; - esac + local section="$1" i IFS=$'\n' + for i in $(git --git-dir="$(__gitdir)" config --name-only --get-regexp "^$section\..*" 2>/dev/null); do + echo "${i#$section.}" done } +__git_pretty_aliases () +{ + __git_get_config_variables "pretty" +} + __git_aliases () { - local i IFS=$'\n' - for i in $(git --git-dir="$(__gitdir)" config --get-regexp "alias\..*" 2>/dev/null); do - case "$i" in - alias.*) - i="${i#alias.}" - echo "${i/ */}" - ;; - esac - done + __git_get_config_variables "alias" } # __git_aliased_command requires 1 argument @@ -774,6 +774,10 @@ __git_aliased_command () -*) : option ;; *=*) : setting env ;; git) : git itself ;; + \(\)) : skip parens of shell function definition ;; + {) : skip start of shell helper function ;; + :) : skip null command ;; + \'*) : skip opening quote after sh -c ;; *) echo "$word" return @@ -902,7 +906,7 @@ _git_add () esac # XXX should we check for --update and --all options ? - __git_complete_index_file "--others --modified" + __git_complete_index_file "--others --modified --directory --no-empty-directory" } _git_archive () @@ -966,7 +970,7 @@ _git_branch () case "$cur" in --set-upstream-to=*) - __gitcomp "$(__git_refs)" "" "${cur##--set-upstream-to=}" + __gitcomp_nl "$(__git_refs)" "" "${cur##--set-upstream-to=}" ;; --*) __gitcomp " @@ -1034,7 +1038,7 @@ _git_checkout () _git_cherry () { - __gitcomp "$(__git_refs)" + __gitcomp_nl "$(__git_refs)" } _git_cherry_pick () @@ -1064,7 +1068,7 @@ _git_clean () esac # XXX should we check for -x option ? - __git_complete_index_file "--others" + __git_complete_index_file "--others --directory" } _git_clone () @@ -1103,7 +1107,7 @@ _git_commit () case "$cur" in --cleanup=*) - __gitcomp "default strip verbatim whitespace + __gitcomp "default scissors strip verbatim whitespace " "" "${cur##--cleanup=}" return ;; @@ -1158,8 +1162,8 @@ __git_diff_common_options="--stat --numstat --shortstat --summary --full-index --binary --abbrev --diff-filter= --find-copies-harder --text --ignore-space-at-eol --ignore-space-change - --ignore-all-space --exit-code --quiet --ext-diff - --no-ext-diff + --ignore-all-space --ignore-blank-lines --exit-code + --quiet --ext-diff --no-ext-diff --no-prefix --src-prefix= --dst-prefix= --inter-hunk-context= --patience --histogram --minimal @@ -1189,8 +1193,8 @@ _git_diff () __git_complete_revlist_file } -__git_mergetools_common="diffuse ecmerge emerge kdiff3 meld opendiff - tkdiff vimdiff gvimdiff xxdiff araxis p4merge bc3 codecompare +__git_mergetools_common="diffuse diffmerge ecmerge emerge kdiff3 meld opendiff + tkdiff vimdiff gvimdiff xxdiff araxis p4merge bc codecompare " _git_difftool () @@ -1214,14 +1218,20 @@ _git_difftool () __git_complete_revlist_file } +__git_fetch_recurse_submodules="yes on-demand no" + __git_fetch_options=" --quiet --verbose --append --upload-pack --force --keep --depth= - --tags --no-tags --all --prune --dry-run + --tags --no-tags --all --prune --dry-run --recurse-submodules= " _git_fetch () { case "$cur" in + --recurse-submodules=*) + __gitcomp "$__git_fetch_recurse_submodules" "" "${cur##--recurse-submodules=}" + return + ;; --*) __gitcomp "$__git_fetch_options" return @@ -1285,7 +1295,7 @@ _git_gitk () } __git_match_ctag() { - awk "/^${1////\\/}/ { print \$1 }" "$2" + awk "/^${1//\//\\/}/ { print \$1 }" "$2" } _git_grep () @@ -1405,7 +1415,7 @@ __git_log_gitk_options=" # Options that go well for log and shortlog (not gitk) __git_log_shortlog_options=" --author= --committer= --grep= - --all-match + --all-match --invert-grep " __git_log_pretty_formats="oneline short medium full fuller email raw format:" @@ -1431,7 +1441,7 @@ _git_log () return ;; --decorate=*) - __gitcomp "long short" "" "${cur##--decorate=}" + __gitcomp "full short no" "" "${cur##--decorate=}" return ;; --*) @@ -1444,6 +1454,7 @@ _git_log () --abbrev-commit --abbrev= --relative-date --date= --pretty= --format= --oneline + --show-signature --cherry-pick --graph --decorate --decorate= @@ -1459,9 +1470,12 @@ _git_log () __git_complete_revlist } +# Common merge options shared by git-merge(1) and git-pull(1). __git_merge_options=" --no-commit --no-stat --log --no-log --squash --strategy --commit --stat --no-squash --ff --no-ff --ff-only --edit --no-edit + --verify-signatures --no-verify-signatures --gpg-sign + --quiet --verbose --progress --no-progress " _git_merge () @@ -1470,7 +1484,8 @@ _git_merge () case "$cur" in --*) - __gitcomp "$__git_merge_options" + __gitcomp "$__git_merge_options + --rerere-autoupdate --no-rerere-autoupdate --abort" return esac __gitcomp_nl "$(__git_refs)" @@ -1492,6 +1507,12 @@ _git_mergetool () _git_merge_base () { + case "$cur" in + --*) + __gitcomp "--octopus --independent --is-ancestor --fork-point" + return + ;; + esac __gitcomp_nl "$(__git_refs)" } @@ -1570,6 +1591,10 @@ _git_pull () __git_complete_strategy && return case "$cur" in + --recurse-submodules=*) + __gitcomp "$__git_fetch_recurse_submodules" "" "${cur##--recurse-submodules=}" + return + ;; --*) __gitcomp " --rebase --no-rebase @@ -1582,22 +1607,55 @@ _git_pull () __git_complete_remote_or_refspec } +__git_push_recurse_submodules="check on-demand" + +__git_complete_force_with_lease () +{ + local cur_=$1 + + case "$cur_" in + --*=) + ;; + *:*) + __gitcomp_nl "$(__git_refs)" "" "${cur_#*:}" + ;; + *) + __gitcomp_nl "$(__git_refs)" "" "$cur_" + ;; + esac +} + _git_push () { case "$prev" in --repo) __gitcomp_nl "$(__git_remotes)" return + ;; + --recurse-submodules) + __gitcomp "$__git_push_recurse_submodules" + return + ;; esac case "$cur" in --repo=*) __gitcomp_nl "$(__git_remotes)" "" "${cur##--repo=}" return ;; + --recurse-submodules=*) + __gitcomp "$__git_push_recurse_submodules" "" "${cur##--recurse-submodules=}" + return + ;; + --force-with-lease=*) + __git_complete_force_with_lease "${cur##--force-with-lease=}" + return + ;; --*) __gitcomp " --all --mirror --tags --dry-run --force --verbose + --quiet --prune --delete --follow-tags --receive-pack= --repo= --set-upstream + --force-with-lease --force-with-lease= --recurse-submodules= " return ;; @@ -1608,7 +1666,10 @@ _git_push () _git_rebase () { local dir="$(__gitdir)" - if [ -d "$dir"/rebase-apply ] || [ -d "$dir"/rebase-merge ]; then + if [ -f "$dir"/rebase-merge/interactive ]; then + __gitcomp "--continue --skip --abort --edit-todo" + return + elif [ -d "$dir"/rebase-apply ] || [ -d "$dir"/rebase-merge ]; then __gitcomp "--continue --skip --abort" return fi @@ -1624,7 +1685,8 @@ _git_rebase () --preserve-merges --stat --no-stat --committer-date-is-author-date --ignore-date --ignore-whitespace --whitespace= - --autosquash + --autosquash --fork-point --no-fork-point + --autostash " return @@ -1714,15 +1776,7 @@ __git_config_get_set_variables () c=$((--c)) done - git --git-dir="$(__gitdir)" config $config_file --list 2>/dev/null | - while read -r line - do - case "$line" in - *.*=*) - echo "${line/=*/}" - ;; - esac - done + git --git-dir="$(__gitdir)" config $config_file --name-only --list 2>/dev/null } _git_config () @@ -1807,6 +1861,10 @@ _git_config () __gitcomp "$__git_send_email_suppresscc_options" return ;; + sendemail.transferencoding) + __gitcomp "7bit 8bit quoted-printable base64" + return + ;; --get|--get-all|--unset|--unset-all) __gitcomp_nl "$(__git_config_get_set_variables)" return @@ -1823,6 +1881,7 @@ _git_config () --get --get-all --get-regexp --add --unset --unset-all --remove-section --rename-section + --name-only " return ;; @@ -1834,6 +1893,7 @@ _git_config () branch.*) local pfx="${cur%.*}." cur_="${cur#*.}" __gitcomp_nl "$(__git_heads)" "$pfx" "$cur_" "." + __gitcomp_nl_append $'autosetupmerge\nautosetuprebase\n' "$pfx" "$cur_" return ;; guitool.*.*) @@ -1876,6 +1936,7 @@ _git_config () remote.*) local pfx="${cur%.*}." cur_="${cur#*.}" __gitcomp_nl "$(__git_remotes)" "$pfx" "$cur_" "." + __gitcomp_nl_append "pushdefault" "$pfx" "$cur_" return ;; url.*.*) @@ -1939,6 +2000,7 @@ _git_config () color.status.changed color.status.header color.status.nobranch + color.status.unmerged color.status.untracked color.status.updated color.ui @@ -1959,7 +2021,6 @@ _git_config () core.fileMode core.fsyncobjectfiles core.gitProxy - core.ignoreCygwinFSTricks core.ignoreStat core.ignorecase core.logAllRefUpdates @@ -1999,6 +2060,7 @@ _git_config () fetch.unpackLimit format.attach format.cc + format.coverLetter format.headers format.numbered format.pretty @@ -2050,6 +2112,8 @@ _git_config () http.noEPSV http.postBuffer http.proxy + http.sslCipherList + http.sslVersion http.sslCAInfo http.sslCAPath http.sslCert @@ -2113,6 +2177,7 @@ _git_config () pull.octopus pull.twohead push.default + push.followTags rebase.autosquash rebase.stat receive.autogc @@ -2186,12 +2251,7 @@ _git_remote () __git_complete_remote_or_refspec ;; update) - local i c='' IFS=$'\n' - for i in $(git --git-dir="$(__gitdir)" config --get-regexp "remotes\..*" 2>/dev/null); do - i="${i#remotes.}" - c="$c ${i/ */}" - done - __gitcomp "$c" + __gitcomp "$(__git_get_config_variables "remotes")" ;; *) ;; @@ -2218,6 +2278,11 @@ _git_reset () _git_revert () { + local dir="$(__gitdir)" + if [ -f "$dir"/REVERT_HEAD ]; then + __gitcomp "--continue --quit --abort" + return + fi case "$cur" in --*) __gitcomp "--edit --mainline --no-edit --no-commit --signoff" @@ -2272,6 +2337,7 @@ _git_show () ;; --*) __gitcomp "--pretty= --format= --abbrev-commit --oneline + --show-signature $__git_diff_common_options " return @@ -2477,6 +2543,16 @@ _git_tag () __gitcomp_nl "$(__git_refs)" ;; esac + + case "$cur" in + --*) + __gitcomp " + --list --delete --verify --annotate --message --file + --sign --cleanup --local-user --force --column --sort + --contains --points-at + " + ;; + esac } _git_whatchanged () @@ -2492,9 +2568,10 @@ __git_main () i="${words[c]}" case "$i" in --git-dir=*) __git_dir="${i#--git-dir=}" ;; + --git-dir) ((c++)) ; __git_dir="${words[c]}" ;; --bare) __git_dir="." ;; --help) command="help"; break ;; - -c) c=$((++c)) ;; + -c|--work-tree|--namespace) ((c++)) ;; -*) ;; *) command="$i"; break ;; esac @@ -2512,6 +2589,7 @@ __git_main () --exec-path --exec-path= --html-path + --man-path --info-path --work-tree= --namespace= @@ -2530,6 +2608,7 @@ __git_main () local expansion=$(__git_aliased_command "$command") if [ -n "$expansion" ]; then + words[1]=$expansion completion_func="_git_${expansion//-/_}" declare -f $completion_func >/dev/null && $completion_func fi @@ -2580,7 +2659,7 @@ if [[ -n ${ZSH_VERSION-} ]]; then --*=*|*.) ;; *) c="$c " ;; esac - array+=("$c") + array[${#array[@]}+1]="$c" done compset -P '*[=:]' compadd -Q -S '' -p "${2-}" -a -- array && _ret=0 diff --git a/contrib/completion/git-completion.tcsh b/contrib/completion/git-completion.tcsh index eaacaf0c3e..4a790d8f4e 100644 --- a/contrib/completion/git-completion.tcsh +++ b/contrib/completion/git-completion.tcsh @@ -1,5 +1,3 @@ -#!tcsh -# # tcsh completion support for core Git. # # Copyright (C) 2012 Marc Khouzam <marc.khouzam@gmail.com> @@ -43,7 +41,7 @@ if ( ! -e ${__git_tcsh_completion_original_script} ) then exit endif -cat << EOF > ${__git_tcsh_completion_script} +cat << EOF >! ${__git_tcsh_completion_script} #!bash # # This script is GENERATED and will be overwritten automatically. diff --git a/contrib/completion/git-completion.zsh b/contrib/completion/git-completion.zsh index fac5e711eb..e25541308a 100644 --- a/contrib/completion/git-completion.zsh +++ b/contrib/completion/git-completion.zsh @@ -9,7 +9,7 @@ # # If your script is somewhere else, you can configure it on your ~/.zshrc: # -# zstyle ':completion:*:*:git:*' script ~/.git-completion.sh +# zstyle ':completion:*:*:git:*' script ~/.git-completion.zsh # # The recommended way to install this script is to copy to '~/.zsh/_git', and # then add the following to your ~/.zshrc file: @@ -30,10 +30,10 @@ if [ -z "$script" ]; then local -a locations local e locations=( + $(dirname ${funcsourcetrace[1]%:*})/git-completion.bash '/etc/bash_completion.d/git' # fedora, old debian '/usr/share/bash-completion/completions/git' # arch, ubuntu, new debian '/usr/share/bash-completion/git' # gentoo - $(dirname ${funcsourcetrace[1]%:*})/git-completion.bash ) for e in $locations; do test -f $e && script="$e" && break @@ -76,6 +76,14 @@ __gitcomp_nl () compadd -Q -S "${4- }" -p "${2-}" -- ${=1} && _ret=0 } +__gitcomp_nl_append () +{ + emulate -L zsh + + local IFS=$'\n' + compadd -Q -S "${4- }" -p "${2-}" -- ${=1} && _ret=0 +} + __gitcomp_file () { emulate -L zsh @@ -96,6 +104,7 @@ __git_zsh_bash_func () local expansion=$(__git_aliased_command "$command") if [ -n "$expansion" ]; then + words[1]=$expansion completion_func="_git_${expansion//-/_}" declare -f $completion_func >/dev/null && $completion_func fi diff --git a/contrib/completion/git-prompt.sh b/contrib/completion/git-prompt.sh index 07a6218d10..07b52bedf1 100644 --- a/contrib/completion/git-prompt.sh +++ b/contrib/completion/git-prompt.sh @@ -3,7 +3,7 @@ # Copyright (C) 2006,2007 Shawn O. Pearce <spearce@spearce.org> # Distributed under the GNU General Public License, version 2.0. # -# This script allows you to see the current branch in your prompt. +# This script allows you to see repository status in your prompt. # # To enable: # @@ -13,24 +13,27 @@ # 3a) Change your PS1 to call __git_ps1 as # command-substitution: # Bash: PS1='[\u@\h \W$(__git_ps1 " (%s)")]\$ ' -# ZSH: PS1='[%n@%m %c$(__git_ps1 " (%s)")]\$ ' +# ZSH: setopt PROMPT_SUBST ; PS1='[%n@%m %c$(__git_ps1 " (%s)")]\$ ' # the optional argument will be used as format string. -# 3b) Alternatively, if you are using bash, __git_ps1 can be -# used for PROMPT_COMMAND with two parameters, <pre> and -# <post>, which are strings you would put in $PS1 before -# and after the status string generated by the git-prompt -# machinery. e.g. +# 3b) Alternatively, for a slightly faster prompt, __git_ps1 can +# be used for PROMPT_COMMAND in Bash or for precmd() in Zsh +# with two parameters, <pre> and <post>, which are strings +# you would put in $PS1 before and after the status string +# generated by the git-prompt machinery. e.g. # Bash: PROMPT_COMMAND='__git_ps1 "\u@\h:\w" "\\\$ "' +# will show username, at-sign, host, colon, cwd, then +# various status string, followed by dollar and SP, as +# your prompt. # ZSH: precmd () { __git_ps1 "%n" ":%~$ " "|%s" } -# will show username, at-sign, host, colon, cwd, then -# various status string, followed by dollar and SP, as -# your prompt. +# will show username, pipe, then various status string, +# followed by colon, cwd, dollar and SP, as your prompt. # Optionally, you can supply a third argument with a printf # format string to finetune the output of the branch status # -# The argument to __git_ps1 will be displayed only if you are currently -# in a git repository. The %s token will be the name of the current -# branch. +# The repository status will be displayed only if you are currently in a +# git repository. The %s token is the placeholder for the shown status. +# +# The prompt status always includes the current branch name. # # In addition, if you set GIT_PS1_SHOWDIRTYSTATE to a nonempty value, # unstaged (*) and staged (+) changes will be shown next to the branch @@ -57,11 +60,16 @@ # of values: # # verbose show number of commits ahead/behind (+/-) upstream +# name if verbose, then also show the upstream abbrev name # legacy don't use the '--count' option available in recent # versions of git-rev-list # git always compare HEAD to @{upstream} # svn always compare HEAD to your SVN upstream # +# You can change the separator between the branch name and the above +# state symbols by setting GIT_PS1_STATESEPARATOR. The default separator +# is SP. +# # By default, __git_ps1 will compare HEAD to your SVN upstream if it can # find one, or @{upstream} otherwise. Once you have set # GIT_PS1_SHOWUPSTREAM, you can override it on a per-repository basis by @@ -78,31 +86,17 @@ # # If you would like a colored hint about the current dirty state, set # GIT_PS1_SHOWCOLORHINTS to a nonempty value. The colors are based on -# the colored output of "git status -sb". +# the colored output of "git status -sb" and are available only when +# using __git_ps1 for PROMPT_COMMAND or precmd. +# +# If you would like __git_ps1 to do nothing in the case when the current +# directory is set up to be ignored by git, then set +# GIT_PS1_HIDE_IF_PWD_IGNORED to a nonempty value. Override this on the +# repository level by setting bash.hideIfPwdIgnored to "false". -# __gitdir accepts 0 or 1 arguments (i.e., location) -# returns location of .git repo -__gitdir () -{ - # Note: this function is duplicated in git-completion.bash - # When updating it, make sure you update the other one to match. - if [ -z "${1-}" ]; then - if [ -n "${__git_dir-}" ]; then - echo "$__git_dir" - elif [ -n "${GIT_DIR-}" ]; then - test -d "${GIT_DIR-}" || return 1 - echo "$GIT_DIR" - elif [ -d .git ]; then - echo .git - else - git rev-parse --git-dir 2>/dev/null - fi - elif [ -d "$1/.git" ]; then - echo "$1/.git" - else - echo "$1" - fi -} +# check whether printf supports -v +__git_printf_supports_v= +printf -v __git_printf_supports_v -- '%s' yes >/dev/null 2>&1 # stores the divergence from upstream in $p # used by GIT_PS1_SHOWUPSTREAM @@ -110,7 +104,7 @@ __git_ps1_show_upstream () { local key value local svn_remote svn_url_pattern count n - local upstream=git legacy="" verbose="" + local upstream=git legacy="" verbose="" name="" svn_remote=() # get some config options from git-config @@ -126,7 +120,7 @@ __git_ps1_show_upstream () ;; svn-remote.*.url) svn_remote[$((${#svn_remote[@]} + 1))]="$value" - svn_url_pattern+="\\|$value" + svn_url_pattern="$svn_url_pattern\\|$value" upstream=svn+git # default upstream is SVN if available, else git ;; esac @@ -138,6 +132,7 @@ __git_ps1_show_upstream () git|svn) upstream="$option" ;; verbose) verbose=1 ;; legacy) legacy=1 ;; + name) name=1 ;; esac done @@ -220,13 +215,25 @@ __git_ps1_show_upstream () *) # diverged from upstream p=" u+${count#* }-${count% *}" ;; esac + if [[ -n "$count" && -n "$name" ]]; then + __git_ps1_upstream_name=$(git rev-parse \ + --abbrev-ref "$upstream" 2>/dev/null) + if [ $pcmode = yes ] && [ $ps1_expanded = yes ]; then + p="$p \${__git_ps1_upstream_name}" + else + p="$p ${__git_ps1_upstream_name}" + # not needed anymore; keep user's + # environment clean + unset __git_ps1_upstream_name + fi + fi fi } # Helper function that is meant to be called from __git_ps1. It -# builds up a gitstring injecting color codes into the appropriate -# places. +# injects color codes into the appropriate gitstring variables used +# to build a gitstring. __git_ps1_colorize_gitstring () { if [[ -n ${ZSH_VERSION-} ]]; then @@ -234,74 +241,47 @@ __git_ps1_colorize_gitstring () local c_green='%F{green}' local c_lblue='%F{blue}' local c_clear='%f' - local bad_color=$c_red - local ok_color=$c_green - local branch_color="$c_clear" - local flags_color="$c_lblue" - local branchstring="$c${b##refs/heads/}" - - if [ $detached = no ]; then - branch_color="$ok_color" - else - branch_color="$bad_color" - fi - - gitstring="$branch_color$branchstring$c_clear" - - if [ -n "$w$i$s$u$r$p" ]; then - gitstring="$gitstring$z" - fi - if [ "$w" = "*" ]; then - gitstring="$gitstring$bad_color$w" - fi - if [ -n "$i" ]; then - gitstring="$gitstring$ok_color$i" - fi - if [ -n "$s" ]; then - gitstring="$gitstring$flags_color$s" - fi - if [ -n "$u" ]; then - gitstring="$gitstring$bad_color$u" - fi - gitstring="$gitstring$c_clear$r$p" - return + else + # Using \[ and \] around colors is necessary to prevent + # issues with command line editing/browsing/completion! + local c_red='\[\e[31m\]' + local c_green='\[\e[32m\]' + local c_lblue='\[\e[1;34m\]' + local c_clear='\[\e[0m\]' fi - local c_red='\e[31m' - local c_green='\e[32m' - local c_lblue='\e[1;34m' - local c_clear='\e[0m' local bad_color=$c_red local ok_color=$c_green - local branch_color="$c_clear" local flags_color="$c_lblue" - local branchstring="$c${b##refs/heads/}" + local branch_color="" if [ $detached = no ]; then branch_color="$ok_color" else branch_color="$bad_color" fi + c="$branch_color$c" - # Setting gitstring directly with \[ and \] around colors - # is necessary to prevent wrapping issues! - gitstring="\[$branch_color\]$branchstring\[$c_clear\]" - - if [ -n "$w$i$s$u$r$p" ]; then - gitstring="$gitstring$z" - fi + z="$c_clear$z" if [ "$w" = "*" ]; then - gitstring="$gitstring\[$bad_color\]$w" + w="$bad_color$w" fi if [ -n "$i" ]; then - gitstring="$gitstring\[$ok_color\]$i" + i="$ok_color$i" fi if [ -n "$s" ]; then - gitstring="$gitstring\[$flags_color\]$s" + s="$flags_color$s" fi if [ -n "$u" ]; then - gitstring="$gitstring\[$bad_color\]$u" + u="$bad_color$u" fi - gitstring="$gitstring\[$c_clear\]$r$p" + r="$c_clear$r" +} + +__git_eread () +{ + local f="$1" + shift + test -r "$f" && read "$@" <"$f" } # __git_ps1 accepts 0 or 1 arguments (i.e., format string) @@ -317,6 +297,8 @@ __git_ps1_colorize_gitstring () # In this mode you can request colored hints using GIT_PS1_SHOWCOLORHINTS=true __git_ps1 () { + # preserve exit status + local exit=$? local pcmode=no local detached=no local ps1pc_start='\u@\h:\w ' @@ -328,57 +310,132 @@ __git_ps1 () ps1pc_start="$1" ps1pc_end="$2" printf_format="${3:-$printf_format}" + # set PS1 to a plain prompt so that we can + # simply return early if the prompt should not + # be decorated + PS1="$ps1pc_start$ps1pc_end" ;; 0|1) printf_format="${1:-$printf_format}" ;; - *) return + *) return $exit ;; esac - local g="$(__gitdir)" - if [ -z "$g" ]; then - if [ $pcmode = yes ]; then - #In PC mode PS1 always needs to be set - PS1="$ps1pc_start$ps1pc_end" + # ps1_expanded: This variable is set to 'yes' if the shell + # subjects the value of PS1 to parameter expansion: + # + # * bash does unless the promptvars option is disabled + # * zsh does not unless the PROMPT_SUBST option is set + # * POSIX shells always do + # + # If the shell would expand the contents of PS1 when drawing + # the prompt, a raw ref name must not be included in PS1. + # This protects the user from arbitrary code execution via + # specially crafted ref names. For example, a ref named + # 'refs/heads/$(IFS=_;cmd=sudo_rm_-rf_/;$cmd)' might cause the + # shell to execute 'sudo rm -rf /' when the prompt is drawn. + # + # Instead, the ref name should be placed in a separate global + # variable (in the __git_ps1_* namespace to avoid colliding + # with the user's environment) and that variable should be + # referenced from PS1. For example: + # + # __git_ps1_foo=$(do_something_to_get_ref_name) + # PS1="...stuff...\${__git_ps1_foo}...stuff..." + # + # If the shell does not expand the contents of PS1, the raw + # ref name must be included in PS1. + # + # The value of this variable is only relevant when in pcmode. + # + # Assume that the shell follows the POSIX specification and + # expands PS1 unless determined otherwise. (This is more + # likely to be correct if the user has a non-bash, non-zsh + # shell and safer than the alternative if the assumption is + # incorrect.) + # + local ps1_expanded=yes + [ -z "$ZSH_VERSION" ] || [[ -o PROMPT_SUBST ]] || ps1_expanded=no + [ -z "$BASH_VERSION" ] || shopt -q promptvars || ps1_expanded=no + + local repo_info rev_parse_exit_code + repo_info="$(git rev-parse --git-dir --is-inside-git-dir \ + --is-bare-repository --is-inside-work-tree \ + --short HEAD 2>/dev/null)" + rev_parse_exit_code="$?" + + if [ -z "$repo_info" ]; then + return $exit + fi + + local short_sha + if [ "$rev_parse_exit_code" = "0" ]; then + short_sha="${repo_info##*$'\n'}" + repo_info="${repo_info%$'\n'*}" + fi + local inside_worktree="${repo_info##*$'\n'}" + repo_info="${repo_info%$'\n'*}" + local bare_repo="${repo_info##*$'\n'}" + repo_info="${repo_info%$'\n'*}" + local inside_gitdir="${repo_info##*$'\n'}" + local g="${repo_info%$'\n'*}" + + if [ "true" = "$inside_worktree" ] && + [ -n "${GIT_PS1_HIDE_IF_PWD_IGNORED-}" ] && + [ "$(git config --bool bash.hideIfPwdIgnored)" != "false" ] && + git check-ignore -q . + then + return $exit + fi + + local r="" + local b="" + local step="" + local total="" + if [ -d "$g/rebase-merge" ]; then + __git_eread "$g/rebase-merge/head-name" b + __git_eread "$g/rebase-merge/msgnum" step + __git_eread "$g/rebase-merge/end" total + if [ -f "$g/rebase-merge/interactive" ]; then + r="|REBASE-i" + else + r="|REBASE-m" fi else - local r="" - local b="" - local step="" - local total="" - if [ -d "$g/rebase-merge" ]; then - b="$(cat "$g/rebase-merge/head-name" 2>/dev/null)" - step=$(cat "$g/rebase-merge/msgnum" 2>/dev/null) - total=$(cat "$g/rebase-merge/end" 2>/dev/null) - if [ -f "$g/rebase-merge/interactive" ]; then - r="|REBASE-i" + if [ -d "$g/rebase-apply" ]; then + __git_eread "$g/rebase-apply/next" step + __git_eread "$g/rebase-apply/last" total + if [ -f "$g/rebase-apply/rebasing" ]; then + __git_eread "$g/rebase-apply/head-name" b + r="|REBASE" + elif [ -f "$g/rebase-apply/applying" ]; then + r="|AM" else - r="|REBASE-m" + r="|AM/REBASE" fi + elif [ -f "$g/MERGE_HEAD" ]; then + r="|MERGING" + elif [ -f "$g/CHERRY_PICK_HEAD" ]; then + r="|CHERRY-PICKING" + elif [ -f "$g/REVERT_HEAD" ]; then + r="|REVERTING" + elif [ -f "$g/BISECT_LOG" ]; then + r="|BISECTING" + fi + + if [ -n "$b" ]; then + : + elif [ -h "$g/HEAD" ]; then + # symlink symbolic ref + b="$(git symbolic-ref HEAD 2>/dev/null)" else - if [ -d "$g/rebase-apply" ]; then - step=$(cat "$g/rebase-apply/next" 2>/dev/null) - total=$(cat "$g/rebase-apply/last" 2>/dev/null) - if [ -f "$g/rebase-apply/rebasing" ]; then - b="$(cat "$g/rebase-apply/head-name" 2>/dev/null)" - r="|REBASE" - elif [ -f "$g/rebase-apply/applying" ]; then - r="|AM" - else - r="|AM/REBASE" - fi - elif [ -f "$g/MERGE_HEAD" ]; then - r="|MERGING" - elif [ -f "$g/CHERRY_PICK_HEAD" ]; then - r="|CHERRY-PICKING" - elif [ -f "$g/REVERT_HEAD" ]; then - r="|REVERTING" - elif [ -f "$g/BISECT_LOG" ]; then - r="|BISECTING" + local head="" + if ! __git_eread "$g/HEAD" head; then + return $exit fi - - test -n "$b" || - b="$(git symbolic-ref HEAD 2>/dev/null)" || { + # is it a symbolic ref? + b="${head#ref: }" + if [ "$head" = "$b" ]; then detached=yes b="$( case "${GIT_PS1_DESCRIBE_STYLE-}" in @@ -392,70 +449,84 @@ __git_ps1 () git describe --tags --exact-match HEAD ;; esac 2>/dev/null)" || - b="$(cut -c1-7 "$g/HEAD" 2>/dev/null)..." || - b="unknown" + b="$short_sha..." b="($b)" - } + fi fi + fi - if [ -n "$step" ] && [ -n "$total" ]; then - r="$r $step/$total" - fi + if [ -n "$step" ] && [ -n "$total" ]; then + r="$r $step/$total" + fi - local w="" - local i="" - local s="" - local u="" - local c="" - local p="" + local w="" + local i="" + local s="" + local u="" + local c="" + local p="" - if [ "true" = "$(git rev-parse --is-inside-git-dir 2>/dev/null)" ]; then - if [ "true" = "$(git rev-parse --is-bare-repository 2>/dev/null)" ]; then - c="BARE:" + if [ "true" = "$inside_gitdir" ]; then + if [ "true" = "$bare_repo" ]; then + c="BARE:" + else + b="GIT_DIR!" + fi + elif [ "true" = "$inside_worktree" ]; then + if [ -n "${GIT_PS1_SHOWDIRTYSTATE-}" ] && + [ "$(git config --bool bash.showDirtyState)" != "false" ] + then + git diff --no-ext-diff --quiet --exit-code || w="*" + if [ -n "$short_sha" ]; then + git diff-index --cached --quiet HEAD -- || i="+" else - b="GIT_DIR!" - fi - elif [ "true" = "$(git rev-parse --is-inside-work-tree 2>/dev/null)" ]; then - if [ -n "${GIT_PS1_SHOWDIRTYSTATE-}" ] && - [ "$(git config --bool bash.showDirtyState)" != "false" ] - then - git diff --no-ext-diff --quiet --exit-code || w="*" - if git rev-parse --quiet --verify HEAD >/dev/null; then - git diff-index --cached --quiet HEAD -- || i="+" - else - i="#" - fi - fi - if [ -n "${GIT_PS1_SHOWSTASHSTATE-}" ]; then - git rev-parse --verify refs/stash >/dev/null 2>&1 && s="$" + i="#" fi + fi + if [ -n "${GIT_PS1_SHOWSTASHSTATE-}" ] && + git rev-parse --verify --quiet refs/stash >/dev/null + then + s="$" + fi - if [ -n "${GIT_PS1_SHOWUNTRACKEDFILES-}" ] && - [ "$(git config --bool bash.showUntrackedFiles)" != "false" ] && - [ -n "$(git ls-files --others --exclude-standard)" ] - then - u="%${ZSH_VERSION+%}" - fi + if [ -n "${GIT_PS1_SHOWUNTRACKEDFILES-}" ] && + [ "$(git config --bool bash.showUntrackedFiles)" != "false" ] && + git ls-files --others --exclude-standard --directory --no-empty-directory --error-unmatch -- ':/*' >/dev/null 2>/dev/null + then + u="%${ZSH_VERSION+%}" + fi - if [ -n "${GIT_PS1_SHOWUPSTREAM-}" ]; then - __git_ps1_show_upstream - fi + if [ -n "${GIT_PS1_SHOWUPSTREAM-}" ]; then + __git_ps1_show_upstream fi + fi - local z="${GIT_PS1_STATESEPARATOR-" "}" - local f="$w$i$s$u" - if [ $pcmode = yes ]; then - local gitstring= - if [ -n "${GIT_PS1_SHOWCOLORHINTS-}" ]; then - __git_ps1_colorize_gitstring - else - gitstring="$c${b##refs/heads/}${f:+$z$f}$r$p" - fi + local z="${GIT_PS1_STATESEPARATOR-" "}" + + # NO color option unless in PROMPT_COMMAND mode + if [ $pcmode = yes ] && [ -n "${GIT_PS1_SHOWCOLORHINTS-}" ]; then + __git_ps1_colorize_gitstring + fi + + b=${b##refs/heads/} + if [ $pcmode = yes ] && [ $ps1_expanded = yes ]; then + __git_ps1_branch_name=$b + b="\${__git_ps1_branch_name}" + fi + + local f="$w$i$s$u" + local gitstring="$c$b${f:+$z$f}$r$p" + + if [ $pcmode = yes ]; then + if [ "${__git_printf_supports_v-}" != yes ]; then gitstring=$(printf -- "$printf_format" "$gitstring") - PS1="$ps1pc_start$gitstring$ps1pc_end" else - # NO color option unless in PROMPT_COMMAND mode - printf -- "$printf_format" "$c${b##refs/heads/}${f:+$z$f}$r$p" + printf -v gitstring -- "$printf_format" "$gitstring" fi + PS1="$ps1pc_start$gitstring$ps1pc_end" + else + printf -- "$printf_format" "$gitstring" fi + + return $exit } diff --git a/contrib/contacts/.gitignore b/contrib/contacts/.gitignore new file mode 100644 index 0000000000..f385ee643c --- /dev/null +++ b/contrib/contacts/.gitignore @@ -0,0 +1,3 @@ +git-contacts.1 +git-contacts.html +git-contacts.xml diff --git a/contrib/contacts/Makefile b/contrib/contacts/Makefile new file mode 100644 index 0000000000..a2990f0dcb --- /dev/null +++ b/contrib/contacts/Makefile @@ -0,0 +1,71 @@ +# The default target of this Makefile is... +all:: + +-include ../../config.mak.autogen +-include ../../config.mak + +prefix ?= /usr/local +gitexecdir ?= $(prefix)/libexec/git-core +mandir ?= $(prefix)/share/man +man1dir ?= $(mandir)/man1 +htmldir ?= $(prefix)/share/doc/git-doc + +../../GIT-VERSION-FILE: FORCE + $(MAKE) -C ../../ GIT-VERSION-FILE + +-include ../../GIT-VERSION-FILE + +# this should be set to a 'standard' bsd-type install program +INSTALL ?= install +RM ?= rm -f + +ASCIIDOC = asciidoc +XMLTO = xmlto + +ifndef SHELL_PATH + SHELL_PATH = /bin/sh +endif +SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH)) + +ASCIIDOC_CONF = ../../Documentation/asciidoc.conf +MANPAGE_XSL = ../../Documentation/manpage-normal.xsl + +GIT_CONTACTS := git-contacts + +GIT_CONTACTS_DOC := git-contacts.1 +GIT_CONTACTS_XML := git-contacts.xml +GIT_CONTACTS_TXT := git-contacts.txt +GIT_CONTACTS_HTML := git-contacts.html + +doc: $(GIT_CONTACTS_DOC) $(GIT_CONTACTS_HTML) + +install: $(GIT_CONTACTS) + $(INSTALL) -d -m 755 $(DESTDIR)$(gitexecdir) + $(INSTALL) -m 755 $(GIT_CONTACTS) $(DESTDIR)$(gitexecdir) + +install-doc: install-man install-html + +install-man: $(GIT_CONTACTS_DOC) + $(INSTALL) -d -m 755 $(DESTDIR)$(man1dir) + $(INSTALL) -m 644 $^ $(DESTDIR)$(man1dir) + +install-html: $(GIT_CONTACTS_HTML) + $(INSTALL) -d -m 755 $(DESTDIR)$(htmldir) + $(INSTALL) -m 644 $^ $(DESTDIR)$(htmldir) + +$(GIT_CONTACTS_DOC): $(GIT_CONTACTS_XML) + $(XMLTO) -m $(MANPAGE_XSL) man $^ + +$(GIT_CONTACTS_XML): $(GIT_CONTACTS_TXT) + $(ASCIIDOC) -b docbook -d manpage -f $(ASCIIDOC_CONF) \ + -agit_version=$(GIT_VERSION) $^ + +$(GIT_CONTACTS_HTML): $(GIT_CONTACTS_TXT) + $(ASCIIDOC) -b xhtml11 -d manpage -f $(ASCIIDOC_CONF) \ + -agit_version=$(GIT_VERSION) $^ + +clean: + $(RM) $(GIT_CONTACTS) + $(RM) *.xml *.html *.1 + +.PHONY: FORCE diff --git a/contrib/contacts/git-contacts b/contrib/contacts/git-contacts new file mode 100755 index 0000000000..dbe2abf277 --- /dev/null +++ b/contrib/contacts/git-contacts @@ -0,0 +1,203 @@ +#!/usr/bin/perl + +# List people who might be interested in a patch. Useful as the argument to +# git-send-email --cc-cmd option, and in other situations. +# +# Usage: git contacts <file | rev-list option> ... + +use strict; +use warnings; +use IPC::Open2; + +my $since = '5-years-ago'; +my $min_percent = 10; +my $labels_rx = qr/Signed-off-by|Reviewed-by|Acked-by|Cc/i; +my %seen; + +sub format_contact { + my ($name, $email) = @_; + return "$name <$email>"; +} + +sub parse_commit { + my ($commit, $data) = @_; + my $contacts = $commit->{contacts}; + my $inbody = 0; + for (split(/^/m, $data)) { + if (not $inbody) { + if (/^author ([^<>]+) <(\S+)> .+$/) { + $contacts->{format_contact($1, $2)} = 1; + } elsif (/^$/) { + $inbody = 1; + } + } elsif (/^$labels_rx:\s+([^<>]+)\s+<(\S+?)>$/o) { + $contacts->{format_contact($1, $2)} = 1; + } + } +} + +sub import_commits { + my ($commits) = @_; + return unless %$commits; + my $pid = open2 my $reader, my $writer, qw(git cat-file --batch); + for my $id (keys(%$commits)) { + print $writer "$id\n"; + my $line = <$reader>; + if ($line =~ /^([0-9a-f]{40}) commit (\d+)/) { + my ($cid, $len) = ($1, $2); + die "expected $id but got $cid\n" unless $id eq $cid; + my $data; + # cat-file emits newline after data, so read len+1 + read $reader, $data, $len + 1; + parse_commit($commits->{$id}, $data); + } + } + close $reader; + close $writer; + waitpid($pid, 0); + die "git-cat-file error: $?\n" if $?; +} + +sub get_blame { + my ($commits, $source, $from, $ranges) = @_; + return unless @$ranges; + open my $f, '-|', + qw(git blame --porcelain -C), + map({"-L$_->[0],+$_->[1]"} @$ranges), + '--since', $since, "$from^", '--', $source or die; + while (<$f>) { + if (/^([0-9a-f]{40}) \d+ \d+ \d+$/) { + my $id = $1; + $commits->{$id} = { id => $id, contacts => {} } + unless $seen{$id}; + $seen{$id} = 1; + } + } + close $f; +} + +sub blame_sources { + my ($sources, $commits) = @_; + for my $s (keys %$sources) { + for my $id (keys %{$sources->{$s}}) { + get_blame($commits, $s, $id, $sources->{$s}{$id}); + } + } +} + +sub scan_patches { + my ($sources, $id, $f) = @_; + my $source; + while (<$f>) { + if (/^From ([0-9a-f]{40}) Mon Sep 17 00:00:00 2001$/) { + $id = $1; + $seen{$id} = 1; + } + next unless $id; + if (m{^--- (?:a/(.+)|/dev/null)$}) { + $source = $1; + } elsif (/^@@ -(\d+)(?:,(\d+))?/ && $source) { + my $len = defined($2) ? $2 : 1; + push @{$sources->{$source}{$id}}, [$1, $len] if $len; + } + } +} + +sub scan_patch_file { + my ($commits, $file) = @_; + open my $f, '<', $file or die "read failure: $file: $!\n"; + scan_patches($commits, undef, $f); + close $f; +} + +sub parse_rev_args { + my @args = @_; + open my $f, '-|', + qw(git rev-parse --revs-only --default HEAD --symbolic), @args + or die; + my @revs; + while (<$f>) { + chomp; + push @revs, $_; + } + close $f; + return @revs if scalar(@revs) != 1; + return "^$revs[0]", 'HEAD' unless $revs[0] =~ /^-/; + return $revs[0], 'HEAD'; +} + +sub scan_rev_args { + my ($commits, $args) = @_; + my @revs = parse_rev_args(@$args); + open my $f, '-|', qw(git rev-list --reverse), @revs or die; + while (<$f>) { + chomp; + my $id = $_; + $seen{$id} = 1; + open my $g, '-|', qw(git show -C --oneline), $id or die; + scan_patches($commits, $id, $g); + close $g; + } + close $f; +} + +sub mailmap_contacts { + my ($contacts) = @_; + my %mapped; + my $pid = open2 my $reader, my $writer, qw(git check-mailmap --stdin); + for my $contact (keys(%$contacts)) { + print $writer "$contact\n"; + my $canonical = <$reader>; + chomp $canonical; + $mapped{$canonical} += $contacts->{$contact}; + } + close $reader; + close $writer; + waitpid($pid, 0); + die "git-check-mailmap error: $?\n" if $?; + return \%mapped; +} + +if (!@ARGV) { + die "No input revisions or patch files\n"; +} + +my (@files, @rev_args); +for (@ARGV) { + if (-e) { + push @files, $_; + } else { + push @rev_args, $_; + } +} + +my %sources; +for (@files) { + scan_patch_file(\%sources, $_); +} +if (@rev_args) { + scan_rev_args(\%sources, \@rev_args) +} + +my $toplevel = `git rev-parse --show-toplevel`; +chomp $toplevel; +chdir($toplevel) or die "chdir failure: $toplevel: $!\n"; + +my %commits; +blame_sources(\%sources, \%commits); +import_commits(\%commits); + +my $contacts = {}; +for my $commit (values %commits) { + for my $contact (keys %{$commit->{contacts}}) { + $contacts->{$contact}++; + } +} +$contacts = mailmap_contacts($contacts); + +my $ncommits = scalar(keys %commits); +for my $contact (keys %$contacts) { + my $percent = $contacts->{$contact} * 100 / $ncommits; + next if $percent < $min_percent; + print "$contact\n"; +} diff --git a/contrib/contacts/git-contacts.txt b/contrib/contacts/git-contacts.txt new file mode 100644 index 0000000000..dd914d1261 --- /dev/null +++ b/contrib/contacts/git-contacts.txt @@ -0,0 +1,94 @@ +git-contacts(1) +=============== + +NAME +---- +git-contacts - List people who might be interested in a set of changes + + +SYNOPSIS +-------- +[verse] +'git contacts' (<patch>|<range>|<rev>)... + + +DESCRIPTION +----------- + +Given a set of changes, specified as patch files or revisions, determine people +who might be interested in those changes. This is done by consulting the +history of each patch or revision hunk to find people mentioned by commits +which touched the lines of files under consideration. + +Input consists of one or more patch files or revision arguments. A revision +argument can be a range or a single `<rev>` which is interpreted as +`<rev>..HEAD`, thus the same revision arguments are accepted as for +linkgit:git-format-patch[1]. Patch files and revision arguments can be combined +in the same invocation. + +This command can be useful for determining the list of people with whom to +discuss proposed changes, or for finding the list of recipients to Cc: when +submitting a patch series via `git send-email`. For the latter case, `git +contacts` can be used as the argument to `git send-email`'s `--cc-cmd` option. + + +DISCUSSION +---------- + +`git blame` is invoked for each hunk in a patch file or revision. For each +commit mentioned by `git blame`, the commit message is consulted for people who +authored, reviewed, signed, acknowledged, or were Cc:'d. Once the list of +participants is known, each person's relevance is computed by considering how +many commits mentioned that person compared with the total number of commits +under consideration. The final output consists only of participants who exceed +a minimum threshold of participation. + + +OUTPUT +------ + +For each person of interest, a single line is output, terminated by a newline. +If the person's name is known, ``Name $$<user@host>$$'' is printed; otherwise +only ``$$<user@host>$$'' is printed. + + +EXAMPLES +-------- + +* Consult patch files: ++ +------------ +$ git contacts feature/*.patch +------------ + +* Revision range: ++ +------------ +$ git contacts R1..R2 +------------ + +* From a single revision to `HEAD`: ++ +------------ +$ git contacts origin +------------ + +* Helper for `git send-email`: ++ +------------ +$ git send-email --cc-cmd='git contacts' feature/*.patch +------------ + + +LIMITATIONS +----------- + +Several conditions controlling a person's significance are currently +hard-coded, such as minimum participation level (10%), blame date-limiting (5 +years), and `-C` level for detecting moved and copied lines (a single `-C`). In +the future, these conditions may become configurable. + + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/contrib/convert-grafts-to-replace-refs.sh b/contrib/convert-grafts-to-replace-refs.sh new file mode 100755 index 0000000000..0cbc917b8c --- /dev/null +++ b/contrib/convert-grafts-to-replace-refs.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +# You should execute this script in the repository where you +# want to convert grafts to replace refs. + +GRAFTS_FILE="${GIT_DIR:-.git}/info/grafts" + +. $(git --exec-path)/git-sh-setup + +test -f "$GRAFTS_FILE" || die "Could not find graft file: '$GRAFTS_FILE'" + +grep '^[^# ]' "$GRAFTS_FILE" | +while read definition +do + if test -n "$definition" + then + echo "Converting: $definition" + git replace --graft $definition || + die "Conversion failed for: $definition" + fi +done + +mv "$GRAFTS_FILE" "$GRAFTS_FILE.bak" || + die "Could not rename '$GRAFTS_FILE' to '$GRAFTS_FILE.bak'" + +echo "Success!" +echo "All the grafts in '$GRAFTS_FILE' have been converted to replace refs!" +echo "The grafts file '$GRAFTS_FILE' has been renamed: '$GRAFTS_FILE.bak'" diff --git a/contrib/convert-objects/git-convert-objects.txt b/contrib/convert-objects/git-convert-objects.txt index 0565d83fc4..f871880cfb 100644 --- a/contrib/convert-objects/git-convert-objects.txt +++ b/contrib/convert-objects/git-convert-objects.txt @@ -26,4 +26,4 @@ Documentation by David Greaves, Junio C Hamano and the git-list <git@vger.kernel GIT --- -Part of the gitlink:git[7] suite +Part of the linkgit:git[7] suite diff --git a/contrib/credential/gnome-keyring/Makefile b/contrib/credential/gnome-keyring/Makefile index e6561d8db6..c3c7c98aa1 100644 --- a/contrib/credential/gnome-keyring/Makefile +++ b/contrib/credential/gnome-keyring/Makefile @@ -8,8 +8,8 @@ CFLAGS = -g -O2 -Wall -include ../../../config.mak.autogen -include ../../../config.mak -INCS:=$(shell pkg-config --cflags gnome-keyring-1) -LIBS:=$(shell pkg-config --libs gnome-keyring-1) +INCS:=$(shell pkg-config --cflags gnome-keyring-1 glib-2.0) +LIBS:=$(shell pkg-config --libs gnome-keyring-1 glib-2.0) SRCS:=$(MAIN).c OBJS:=$(SRCS:.c=.o) diff --git a/contrib/credential/gnome-keyring/git-credential-gnome-keyring.c b/contrib/credential/gnome-keyring/git-credential-gnome-keyring.c index f2cdefee60..2a317fca44 100644 --- a/contrib/credential/gnome-keyring/git-credential-gnome-keyring.c +++ b/contrib/credential/gnome-keyring/git-credential-gnome-keyring.c @@ -25,135 +25,148 @@ #include <stdio.h> #include <string.h> -#include <stdarg.h> #include <stdlib.h> -#include <errno.h> +#include <glib.h> #include <gnome-keyring.h> -/* - * This credential struct and API is simplified from git's credential.{h,c} - */ -struct credential -{ - char *protocol; - char *host; - unsigned short port; - char *path; - char *username; - char *password; -}; +#ifdef GNOME_KEYRING_DEFAULT -#define CREDENTIAL_INIT \ - { NULL,NULL,0,NULL,NULL,NULL } + /* Modern gnome-keyring */ -void credential_init(struct credential *c); -void credential_clear(struct credential *c); -int credential_read(struct credential *c); -void credential_write(const struct credential *c); +#include <gnome-keyring-memory.h> -typedef int (*credential_op_cb)(struct credential*); +#else -struct credential_operation -{ - char *name; - credential_op_cb op; -}; + /* + * Support ancient gnome-keyring, circ. RHEL 5.X. + * GNOME_KEYRING_DEFAULT seems to have been introduced with Gnome 2.22, + * and the other features roughly around Gnome 2.20, 6 months before. + * Ubuntu 8.04 used Gnome 2.22 (I think). Not sure any distro used 2.20. + * So the existence/non-existence of GNOME_KEYRING_DEFAULT seems like + * a decent thing to use as an indicator. + */ -#define CREDENTIAL_OP_END \ - { NULL,NULL } +#define GNOME_KEYRING_DEFAULT NULL /* - * Table with operation callbacks is defined in concrete - * credential helper implementation and contains entries - * like { "get", function_to_get_credential } terminated - * by CREDENTIAL_OP_END. + * ancient gnome-keyring returns DENIED when an entry is not found. + * Setting NO_MATCH to DENIED will prevent us from reporting DENIED + * errors during get and erase operations, but we will still report + * DENIED errors during a store. */ -struct credential_operation const credential_helper_ops[]; +#define GNOME_KEYRING_RESULT_NO_MATCH GNOME_KEYRING_RESULT_DENIED -/* ---------------- common helper functions ----------------- */ +#define gnome_keyring_memory_alloc g_malloc +#define gnome_keyring_memory_free gnome_keyring_free_password +#define gnome_keyring_memory_strdup g_strdup -static inline void free_password(char *password) +static const char *gnome_keyring_result_to_message(GnomeKeyringResult result) { - char *c = password; - if (!password) - return; - - while (*c) *c++ = '\0'; - free(password); + switch (result) { + case GNOME_KEYRING_RESULT_OK: + return "OK"; + case GNOME_KEYRING_RESULT_DENIED: + return "Denied"; + case GNOME_KEYRING_RESULT_NO_KEYRING_DAEMON: + return "No Keyring Daemon"; + case GNOME_KEYRING_RESULT_ALREADY_UNLOCKED: + return "Already UnLocked"; + case GNOME_KEYRING_RESULT_NO_SUCH_KEYRING: + return "No Such Keyring"; + case GNOME_KEYRING_RESULT_BAD_ARGUMENTS: + return "Bad Arguments"; + case GNOME_KEYRING_RESULT_IO_ERROR: + return "IO Error"; + case GNOME_KEYRING_RESULT_CANCELLED: + return "Cancelled"; + case GNOME_KEYRING_RESULT_ALREADY_EXISTS: + return "Already Exists"; + default: + return "Unknown Error"; + } } -static inline void warning(const char *fmt, ...) +/* + * Support really ancient gnome-keyring, circ. RHEL 4.X. + * Just a guess for the Glib version. Glib 2.8 was roughly Gnome 2.12 ? + * Which was released with gnome-keyring 0.4.3 ?? + */ +#if GLIB_MAJOR_VERSION == 2 && GLIB_MINOR_VERSION < 8 + +static void gnome_keyring_done_cb(GnomeKeyringResult result, gpointer user_data) { - va_list ap; + gpointer *data = (gpointer *)user_data; + int *done = (int *)data[0]; + GnomeKeyringResult *r = (GnomeKeyringResult *)data[1]; - va_start(ap, fmt); - fprintf(stderr, "warning: "); - vfprintf(stderr, fmt, ap); - fprintf(stderr, "\n" ); - va_end(ap); + *r = result; + *done = 1; } -static inline void error(const char *fmt, ...) +static void wait_for_request_completion(int *done) { - va_list ap; - - va_start(ap, fmt); - fprintf(stderr, "error: "); - vfprintf(stderr, fmt, ap); - fprintf(stderr, "\n" ); - va_end(ap); + GMainContext *mc = g_main_context_default(); + while (!*done) + g_main_context_iteration(mc, TRUE); } -static inline void die(const char *fmt, ...) +static GnomeKeyringResult gnome_keyring_item_delete_sync(const char *keyring, guint32 id) { - va_list ap; + int done = 0; + GnomeKeyringResult result; + gpointer data[] = { &done, &result }; - va_start(ap,fmt); - error(fmt, ap); - va_end(ap); - exit(EXIT_FAILURE); -} + gnome_keyring_item_delete(keyring, id, gnome_keyring_done_cb, data, + NULL); -static inline void die_errno(int err) -{ - error("%s", strerror(err)); - exit(EXIT_FAILURE); + wait_for_request_completion(&done); + + return result; } -static inline char *xstrdup(const char *str) -{ - char *ret = strdup(str); - if (!ret) - die_errno(errno); +#endif +#endif - return ret; -} +/* + * This credential struct and API is simplified from git's credential.{h,c} + */ +struct credential { + char *protocol; + char *host; + unsigned short port; + char *path; + char *username; + char *password; +}; + +#define CREDENTIAL_INIT { NULL, NULL, 0, NULL, NULL, NULL } + +typedef int (*credential_op_cb)(struct credential *); + +struct credential_operation { + char *name; + credential_op_cb op; +}; + +#define CREDENTIAL_OP_END { NULL, NULL } /* ----------------- GNOME Keyring functions ----------------- */ /* create a special keyring option string, if path is given */ -static char* keyring_object(struct credential *c) +static char *keyring_object(struct credential *c) { - char* object = NULL; - if (!c->path) - return object; + return NULL; - object = (char*) malloc(strlen(c->host)+strlen(c->path)+8); - if(!object) - die_errno(errno); + if (c->port) + return g_strdup_printf("%s:%hd/%s", c->host, c->port, c->path); - if(c->port) - sprintf(object,"%s:%hd/%s",c->host,c->port,c->path); - else - sprintf(object,"%s/%s",c->host,c->path); - - return object; + return g_strdup_printf("%s/%s", c->host, c->path); } -int keyring_get(struct credential *c) +static int keyring_get(struct credential *c) { - char* object = NULL; + char *object = NULL; GList *entries; GnomeKeyringNetworkPasswordData *password_data; GnomeKeyringResult result; @@ -173,7 +186,7 @@ int keyring_get(struct credential *c) c->port, &entries); - free(object); + g_free(object); if (result == GNOME_KEYRING_RESULT_NO_MATCH) return EXIT_SUCCESS; @@ -182,18 +195,18 @@ int keyring_get(struct credential *c) return EXIT_SUCCESS; if (result != GNOME_KEYRING_RESULT_OK) { - error("%s",gnome_keyring_result_to_message(result)); + g_critical("%s", gnome_keyring_result_to_message(result)); return EXIT_FAILURE; } /* pick the first one from the list */ - password_data = (GnomeKeyringNetworkPasswordData *) entries->data; + password_data = (GnomeKeyringNetworkPasswordData *)entries->data; - free_password(c->password); - c->password = xstrdup(password_data->password); + gnome_keyring_memory_free(c->password); + c->password = gnome_keyring_memory_strdup(password_data->password); if (!c->username) - c->username = xstrdup(password_data->user); + c->username = g_strdup(password_data->user); gnome_keyring_network_password_list_free(entries); @@ -201,10 +214,11 @@ int keyring_get(struct credential *c) } -int keyring_store(struct credential *c) +static int keyring_store(struct credential *c) { guint32 item_id; - char *object = NULL; + char *object = NULL; + GnomeKeyringResult result; /* * Sanity check that what we are storing is actually sensible. @@ -219,7 +233,7 @@ int keyring_store(struct credential *c) object = keyring_object(c); - gnome_keyring_set_network_password_sync( + result = gnome_keyring_set_network_password_sync( GNOME_KEYRING_DEFAULT, c->username, NULL /* domain */, @@ -231,13 +245,20 @@ int keyring_store(struct credential *c) c->password, &item_id); - free(object); + g_free(object); + + if (result != GNOME_KEYRING_RESULT_OK && + result != GNOME_KEYRING_RESULT_CANCELLED) { + g_critical("%s", gnome_keyring_result_to_message(result)); + return EXIT_FAILURE; + } + return EXIT_SUCCESS; } -int keyring_erase(struct credential *c) +static int keyring_erase(struct credential *c) { - char *object = NULL; + char *object = NULL; GList *entries; GnomeKeyringNetworkPasswordData *password_data; GnomeKeyringResult result; @@ -265,7 +286,7 @@ int keyring_erase(struct credential *c) c->port, &entries); - free(object); + g_free(object); if (result == GNOME_KEYRING_RESULT_NO_MATCH) return EXIT_SUCCESS; @@ -273,23 +294,21 @@ int keyring_erase(struct credential *c) if (result == GNOME_KEYRING_RESULT_CANCELLED) return EXIT_SUCCESS; - if (result != GNOME_KEYRING_RESULT_OK) - { - error("%s",gnome_keyring_result_to_message(result)); + if (result != GNOME_KEYRING_RESULT_OK) { + g_critical("%s", gnome_keyring_result_to_message(result)); return EXIT_FAILURE; } /* pick the first one from the list (delete all matches?) */ - password_data = (GnomeKeyringNetworkPasswordData *) entries->data; + password_data = (GnomeKeyringNetworkPasswordData *)entries->data; result = gnome_keyring_item_delete_sync( password_data->keyring, password_data->item_id); gnome_keyring_network_password_list_free(entries); - if (result != GNOME_KEYRING_RESULT_OK) - { - error("%s",gnome_keyring_result_to_message(result)); + if (result != GNOME_KEYRING_RESULT_OK) { + g_critical("%s", gnome_keyring_result_to_message(result)); return EXIT_FAILURE; } @@ -300,9 +319,8 @@ int keyring_erase(struct credential *c) * Table with helper operation callbacks, used by generic * credential helper main function. */ -struct credential_operation const credential_helper_ops[] = -{ - { "get", keyring_get }, +static struct credential_operation const credential_helper_ops[] = { + { "get", keyring_get }, { "store", keyring_store }, { "erase", keyring_erase }, CREDENTIAL_OP_END @@ -310,67 +328,70 @@ struct credential_operation const credential_helper_ops[] = /* ------------------ credential functions ------------------ */ -void credential_init(struct credential *c) +static void credential_init(struct credential *c) { memset(c, 0, sizeof(*c)); } -void credential_clear(struct credential *c) +static void credential_clear(struct credential *c) { - free(c->protocol); - free(c->host); - free(c->path); - free(c->username); - free_password(c->password); + g_free(c->protocol); + g_free(c->host); + g_free(c->path); + g_free(c->username); + gnome_keyring_memory_free(c->password); credential_init(c); } -int credential_read(struct credential *c) +static int credential_read(struct credential *c) { - char buf[1024]; - ssize_t line_len = 0; - char *key = buf; - char *value; + char *buf; + size_t line_len; + char *key; + char *value; - while (fgets(buf, sizeof(buf), stdin)) - { + key = buf = gnome_keyring_memory_alloc(1024); + + while (fgets(buf, 1024, stdin)) { line_len = strlen(buf); - if(buf[line_len-1]=='\n') - buf[--line_len]='\0'; + if (line_len && buf[line_len-1] == '\n') + buf[--line_len] = '\0'; - if(!line_len) + if (!line_len) break; - value = strchr(buf,'='); - if(!value) { - warning("invalid credential line: %s", key); + value = strchr(buf, '='); + if (!value) { + g_warning("invalid credential line: %s", key); + gnome_keyring_memory_free(buf); return -1; } *value++ = '\0'; if (!strcmp(key, "protocol")) { - free(c->protocol); - c->protocol = xstrdup(value); + g_free(c->protocol); + c->protocol = g_strdup(value); } else if (!strcmp(key, "host")) { - free(c->host); - c->host = xstrdup(value); - value = strrchr(c->host,':'); + g_free(c->host); + c->host = g_strdup(value); + value = strrchr(c->host, ':'); if (value) { *value++ = '\0'; c->port = atoi(value); } } else if (!strcmp(key, "path")) { - free(c->path); - c->path = xstrdup(value); + g_free(c->path); + c->path = g_strdup(value); } else if (!strcmp(key, "username")) { - free(c->username); - c->username = xstrdup(value); + g_free(c->username); + c->username = g_strdup(value); } else if (!strcmp(key, "password")) { - free_password(c->password); - c->password = xstrdup(value); - while (*value) *value++ = '\0'; + gnome_keyring_memory_free(c->password); + c->password = gnome_keyring_memory_strdup(value); + while (*value) + *value++ = '\0'; } /* * Ignore other lines; we don't know what they mean, but @@ -378,17 +399,20 @@ int credential_read(struct credential *c) * learn new lines, and the helpers are updated to match. */ } + + gnome_keyring_memory_free(buf); + return 0; } -void credential_write_item(FILE *fp, const char *key, const char *value) +static void credential_write_item(FILE *fp, const char *key, const char *value) { if (!value) return; fprintf(fp, "%s=%s\n", key, value); } -void credential_write(const struct credential *c) +static void credential_write(const struct credential *c) { /* only write username/password, if set */ credential_write_item(stdout, "username", c->username); @@ -398,16 +422,16 @@ void credential_write(const struct credential *c) static void usage(const char *name) { struct credential_operation const *try_op = credential_helper_ops; - const char *basename = strrchr(name,'/'); + const char *basename = strrchr(name, '/'); basename = (basename) ? basename + 1 : name; fprintf(stderr, "usage: %s <", basename); - while(try_op->name) { - fprintf(stderr,"%s",(try_op++)->name); - if(try_op->name) - fprintf(stderr,"%s","|"); + while (try_op->name) { + fprintf(stderr, "%s", (try_op++)->name); + if (try_op->name) + fprintf(stderr, "%s", "|"); } - fprintf(stderr,"%s",">\n"); + fprintf(stderr, "%s", ">\n"); } int main(int argc, char *argv[]) @@ -415,23 +439,25 @@ int main(int argc, char *argv[]) int ret = EXIT_SUCCESS; struct credential_operation const *try_op = credential_helper_ops; - struct credential cred = CREDENTIAL_INIT; + struct credential cred = CREDENTIAL_INIT; if (!argv[1]) { usage(argv[0]); - goto out; + exit(EXIT_FAILURE); } + g_set_application_name("Git Credential Helper"); + /* lookup operation callback */ - while(try_op->name && strcmp(argv[1], try_op->name)) + while (try_op->name && strcmp(argv[1], try_op->name)) try_op++; /* unsupported operation given -- ignore silently */ - if(!try_op->name || !try_op->op) + if (!try_op->name || !try_op->op) goto out; ret = credential_read(&cred); - if(ret) + if (ret) goto out; /* perform credential operation */ diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc index 6c51c43885..1571a7b269 100755 --- a/contrib/credential/netrc/git-credential-netrc +++ b/contrib/credential/netrc/git-credential-netrc @@ -369,7 +369,9 @@ sub find_netrc_entry { { my $entry_text = join ', ', map { "$_=$entry->{$_}" } keys %$entry; foreach my $check (sort keys %$query) { - if (defined $query->{$check}) { + if (!defined $entry->{$check}) { + log_debug("OK: entry has no $check token, so any value satisfies check $check"); + } elsif (defined $query->{$check}) { log_debug("compare %s [%s] to [%s] (entry: %s)", $check, $entry->{$check}, diff --git a/contrib/credential/wincred/Makefile b/contrib/credential/wincred/Makefile index bad45ca47a..6e992c0866 100644 --- a/contrib/credential/wincred/Makefile +++ b/contrib/credential/wincred/Makefile @@ -1,14 +1,22 @@ all: git-credential-wincred.exe -CC = gcc -RM = rm -f -CFLAGS = -O2 -Wall - -include ../../../config.mak.autogen -include ../../../config.mak +CC ?= gcc +RM ?= rm -f +CFLAGS ?= -O2 -Wall + +prefix ?= /usr/local +libexecdir ?= $(prefix)/libexec/git-core + +INSTALL ?= install + git-credential-wincred.exe : git-credential-wincred.c $(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@ +install: git-credential-wincred.exe + $(INSTALL) -m 755 $^ $(libexecdir) + clean: $(RM) git-credential-wincred.exe diff --git a/contrib/credential/wincred/git-credential-wincred.c b/contrib/credential/wincred/git-credential-wincred.c index a1d38f035b..006134043a 100644 --- a/contrib/credential/wincred/git-credential-wincred.c +++ b/contrib/credential/wincred/git-credential-wincred.c @@ -111,14 +111,23 @@ static void write_item(const char *what, LPCWSTR wbuf, int wlen) * Match an (optional) expected string and a delimiter in the target string, * consuming the matched text by updating the target pointer. */ -static int match_part(LPCWSTR *ptarget, LPCWSTR want, LPCWSTR delim) + +static LPCWSTR wcsstr_last(LPCWSTR str, LPCWSTR find) +{ + LPCWSTR res = NULL, pos; + for (pos = wcsstr(str, find); pos; pos = wcsstr(pos + 1, find)) + res = pos; + return res; +} + +static int match_part_with_last(LPCWSTR *ptarget, LPCWSTR want, LPCWSTR delim, int last) { LPCWSTR delim_pos, start = *ptarget; int len; /* find start of delimiter (or end-of-string if delim is empty) */ if (*delim) - delim_pos = wcsstr(start, delim); + delim_pos = last ? wcsstr_last(start, delim) : wcsstr(start, delim); else delim_pos = start + wcslen(start); @@ -138,6 +147,16 @@ static int match_part(LPCWSTR *ptarget, LPCWSTR want, LPCWSTR delim) return !want || (!wcsncmp(want, start, len) && !want[len]); } +static int match_part(LPCWSTR *ptarget, LPCWSTR want, LPCWSTR delim) +{ + return match_part_with_last(ptarget, want, delim, 0); +} + +static int match_part_last(LPCWSTR *ptarget, LPCWSTR want, LPCWSTR delim) +{ + return match_part_with_last(ptarget, want, delim, 1); +} + static int match_cred(const CREDENTIALW *cred) { LPCWSTR target = cred->TargetName; @@ -146,7 +165,7 @@ static int match_cred(const CREDENTIALW *cred) return match_part(&target, L"git", L":") && match_part(&target, protocol, L"://") && - match_part(&target, wusername, L"@") && + match_part_last(&target, wusername, L"@") && match_part(&target, host, L"/") && match_part(&target, path, L""); } diff --git a/contrib/diff-highlight/README b/contrib/diff-highlight/README index 502e03b305..836b97a730 100644 --- a/contrib/diff-highlight/README +++ b/contrib/diff-highlight/README @@ -58,6 +58,47 @@ following in your git configuration: diff = diff-highlight | less --------------------------------------------- + +Color Config +------------ + +You can configure the highlight colors and attributes using git's +config. The colors for "old" and "new" lines can be specified +independently. There are two "modes" of configuration: + + 1. You can specify a "highlight" color and a matching "reset" color. + This will retain any existing colors in the diff, and apply the + "highlight" and "reset" colors before and after the highlighted + portion. + + 2. You can specify a "normal" color and a "highlight" color. In this + case, existing colors are dropped from that line. The non-highlighted + bits of the line get the "normal" color, and the highlights get the + "highlight" color. + +If no "new" colors are specified, they default to the "old" colors. If +no "old" colors are specified, the default is to reverse the foreground +and background for highlighted portions. + +Examples: + +--------------------------------------------- +# Underline highlighted portions +[color "diff-highlight"] +oldHighlight = ul +oldReset = noul +--------------------------------------------- + +--------------------------------------------- +# Varying background intensities +[color "diff-highlight"] +oldNormal = "black #f8cbcb" +oldHighlight = "black #ffaaaa" +newNormal = "black #cbeecb" +newHighlight = "black #aaffaa" +--------------------------------------------- + + Bugs ---- diff --git a/contrib/diff-highlight/diff-highlight b/contrib/diff-highlight/diff-highlight index c4404d49c9..ffefc31a98 100755 --- a/contrib/diff-highlight/diff-highlight +++ b/contrib/diff-highlight/diff-highlight @@ -1,12 +1,23 @@ #!/usr/bin/perl +use 5.008; use warnings FATAL => 'all'; use strict; # Highlight by reversing foreground and background. You could do # other things like bold or underline if you prefer. -my $HIGHLIGHT = "\x1b[7m"; -my $UNHIGHLIGHT = "\x1b[27m"; +my @OLD_HIGHLIGHT = ( + color_config('color.diff-highlight.oldnormal'), + color_config('color.diff-highlight.oldhighlight', "\x1b[7m"), + color_config('color.diff-highlight.oldreset', "\x1b[27m") +); +my @NEW_HIGHLIGHT = ( + color_config('color.diff-highlight.newnormal', $OLD_HIGHLIGHT[0]), + color_config('color.diff-highlight.newhighlight', $OLD_HIGHLIGHT[1]), + color_config('color.diff-highlight.newreset', $OLD_HIGHLIGHT[2]) +); + +my $RESET = "\x1b[m"; my $COLOR = qr/\x1b\[[0-9;]*m/; my $BORING = qr/$COLOR|\s/; @@ -14,6 +25,10 @@ my @removed; my @added; my $in_hunk; +# Some scripts may not realize that SIGPIPE is being ignored when launching the +# pager--for instance scripts written in Python. +$SIG{PIPE} = 'DEFAULT'; + while (<>) { if (!$in_hunk) { print; @@ -53,6 +68,17 @@ show_hunk(\@removed, \@added); exit 0; +# Ideally we would feed the default as a human-readable color to +# git-config as the fallback value. But diff-highlight does +# not otherwise depend on git at all, and there are reports +# of it being used in other settings. Let's handle our own +# fallback, which means we will work even if git can't be run. +sub color_config { + my ($key, $default) = @_; + my $s = `git config --get-color $key 2>/dev/null`; + return length($s) ? $s : $default; +} + sub show_hunk { my ($a, $b) = @_; @@ -128,8 +154,8 @@ sub highlight_pair { } if (is_pair_interesting(\@a, $pa, $sa, \@b, $pb, $sb)) { - return highlight_line(\@a, $pa, $sa), - highlight_line(\@b, $pb, $sb); + return highlight_line(\@a, $pa, $sa, \@OLD_HIGHLIGHT), + highlight_line(\@b, $pb, $sb, \@NEW_HIGHLIGHT); } else { return join('', @a), @@ -139,20 +165,39 @@ sub highlight_pair { sub split_line { local $_ = shift; - return map { /$COLOR/ ? $_ : (split //) } - split /($COLOR*)/; + return utf8::decode($_) ? + map { utf8::encode($_); $_ } + map { /$COLOR/ ? $_ : (split //) } + split /($COLOR+)/ : + map { /$COLOR/ ? $_ : (split //) } + split /($COLOR+)/; } sub highlight_line { - my ($line, $prefix, $suffix) = @_; - - return join('', - @{$line}[0..($prefix-1)], - $HIGHLIGHT, - @{$line}[$prefix..$suffix], - $UNHIGHLIGHT, - @{$line}[($suffix+1)..$#$line] - ); + my ($line, $prefix, $suffix, $theme) = @_; + + my $start = join('', @{$line}[0..($prefix-1)]); + my $mid = join('', @{$line}[$prefix..$suffix]); + my $end = join('', @{$line}[($suffix+1)..$#$line]); + + # If we have a "normal" color specified, then take over the whole line. + # Otherwise, we try to just manipulate the highlighted bits. + if (defined $theme->[0]) { + s/$COLOR//g for ($start, $mid, $end); + chomp $end; + return join('', + $theme->[0], $start, $RESET, + $theme->[1], $mid, $RESET, + $theme->[0], $end, $RESET, + "\n" + ); + } else { + return join('', + $start, + $theme->[1], $mid, $theme->[2], + $end + ); + } } # Pairs are interesting to highlight only if we are going to end up diff --git a/contrib/diffall/README b/contrib/diffall/README deleted file mode 100644 index 507f17dcd6..0000000000 --- a/contrib/diffall/README +++ /dev/null @@ -1,31 +0,0 @@ -The git-diffall script provides a directory based diff mechanism -for git. - -To determine what diff viewer is used, the script requires either -the 'diff.tool' or 'merge.tool' configuration option to be set. - -This script is compatible with most common forms used to specify a -range of revisions to diff: - - 1. git diffall: shows diff between working tree and staged changes - 2. git diffall --cached [<commit>]: shows diff between staged - changes and HEAD (or other named commit) - 3. git diffall <commit>: shows diff between working tree and named - commit - 4. git diffall <commit> <commit>: show diff between two named commits - 5. git diffall <commit>..<commit>: same as above - 6. git diffall <commit>...<commit>: show the changes on the branch - containing and up to the second, starting at a common ancestor - of both <commit> - -Note: all forms take an optional path limiter [-- <path>*] - -The '--extcmd=<command>' option allows the user to specify a custom -command for viewing diffs. When given, configured defaults are -ignored and the script runs $command $LOCAL $REMOTE. Additionally, -$BASE is set in the environment. - -This script is based on an example provided by Thomas Rast on the -Git list [1]: - -[1] http://thread.gmane.org/gmane.comp.version-control.git/124807 diff --git a/contrib/diffall/git-diffall b/contrib/diffall/git-diffall deleted file mode 100755 index 84f2b654d7..0000000000 --- a/contrib/diffall/git-diffall +++ /dev/null @@ -1,257 +0,0 @@ -#!/bin/sh -# Copyright 2010 - 2012, Tim Henigan <tim.henigan@gmail.com> -# -# Perform a directory diff between commits in the repository using -# the external diff or merge tool specified in the user's config. - -USAGE='[--cached] [--copy-back] [-x|--extcmd=<command>] <commit>{0,2} [-- <path>*] - - --cached Compare to the index rather than the working tree. - - --copy-back Copy files back to the working tree when the diff - tool exits (in case they were modified by the - user). This option is only valid if the diff - compared with the working tree. - - -x=<command> - --extcmd=<command> Specify a custom command for viewing diffs. - git-diffall ignores the configured defaults and - runs $command $LOCAL $REMOTE when this option is - specified. Additionally, $BASE is set in the - environment. -' - -SUBDIRECTORY_OK=1 -. "$(git --exec-path)/git-sh-setup" - -TOOL_MODE=diff -. "$(git --exec-path)/git-mergetool--lib" - -merge_tool="$(get_merge_tool)" -if test -z "$merge_tool" -then - echo "Error: Either the 'diff.tool' or 'merge.tool' option must be set." - usage -fi - -start_dir=$(pwd) - -# All the file paths returned by the diff command are relative to the root -# of the working copy. So if the script is called from a subdirectory, it -# must switch to the root of working copy before trying to use those paths. -cdup=$(git rev-parse --show-cdup) && -cd "$cdup" || { - echo >&2 "Cannot chdir to $cdup, the toplevel of the working tree" - exit 1 -} - -# set up temp dir -tmp=$(perl -e 'use File::Temp qw(tempdir); - $t=tempdir("/tmp/git-diffall.XXXXX") or exit(1); - print $t') || exit 1 -trap 'rm -rf "$tmp"' EXIT - -left= -right= -paths= -dashdash_seen= -compare_staged= -merge_base= -left_dir= -right_dir= -diff_tool= -copy_back= - -while test $# != 0 -do - case "$1" in - -h|--h|--he|--hel|--help) - usage - ;; - --cached) - compare_staged=1 - ;; - --copy-back) - copy_back=1 - ;; - -x|--e|--ex|--ext|--extc|--extcm|--extcmd) - if test $# = 1 - then - echo You must specify the tool for use with --extcmd - usage - else - diff_tool=$2 - shift - fi - ;; - --) - dashdash_seen=1 - ;; - -*) - echo Invalid option: "$1" - usage - ;; - *) - # could be commit, commit range or path limiter - case "$1" in - *...*) - left=${1%...*} - right=${1#*...} - merge_base=1 - ;; - *..*) - left=${1%..*} - right=${1#*..} - ;; - *) - if test -n "$dashdash_seen" - then - paths="$paths$1 " - elif test -z "$left" - then - left=$1 - elif test -z "$right" - then - right=$1 - else - paths="$paths$1 " - fi - ;; - esac - ;; - esac - shift -done - -# Determine the set of files which changed -if test -n "$left" && test -n "$right" -then - left_dir="cmt-$(git rev-parse --short $left)" - right_dir="cmt-$(git rev-parse --short $right)" - - if test -n "$compare_staged" - then - usage - elif test -n "$merge_base" - then - git diff --name-only "$left"..."$right" -- $paths >"$tmp/filelist" - else - git diff --name-only "$left" "$right" -- $paths >"$tmp/filelist" - fi -elif test -n "$left" -then - left_dir="cmt-$(git rev-parse --short $left)" - - if test -n "$compare_staged" - then - right_dir="staged" - git diff --name-only --cached "$left" -- $paths >"$tmp/filelist" - else - right_dir="working_tree" - git diff --name-only "$left" -- $paths >"$tmp/filelist" - fi -else - left_dir="HEAD" - - if test -n "$compare_staged" - then - right_dir="staged" - git diff --name-only --cached -- $paths >"$tmp/filelist" - else - right_dir="working_tree" - git diff --name-only -- $paths >"$tmp/filelist" - fi -fi - -# Exit immediately if there are no diffs -if test ! -s "$tmp/filelist" -then - exit 0 -fi - -if test -n "$copy_back" && test "$right_dir" != "working_tree" -then - echo "--copy-back is only valid when diff includes the working tree." - exit 1 -fi - -# Create the named tmp directories that will hold the files to be compared -mkdir -p "$tmp/$left_dir" "$tmp/$right_dir" - -# Populate the tmp/right_dir directory with the files to be compared -while read name -do - if test -n "$right" - then - ls_list=$(git ls-tree $right "$name") - if test -n "$ls_list" - then - mkdir -p "$tmp/$right_dir/$(dirname "$name")" - git show "$right":"$name" >"$tmp/$right_dir/$name" || true - fi - elif test -n "$compare_staged" - then - ls_list=$(git ls-files -- "$name") - if test -n "$ls_list" - then - mkdir -p "$tmp/$right_dir/$(dirname "$name")" - git show :"$name" >"$tmp/$right_dir/$name" - fi - else - if test -e "$name" - then - mkdir -p "$tmp/$right_dir/$(dirname "$name")" - cp "$name" "$tmp/$right_dir/$name" - fi - fi -done < "$tmp/filelist" - -# Populate the tmp/left_dir directory with the files to be compared -while read name -do - if test -n "$left" - then - ls_list=$(git ls-tree $left "$name") - if test -n "$ls_list" - then - mkdir -p "$tmp/$left_dir/$(dirname "$name")" - git show "$left":"$name" >"$tmp/$left_dir/$name" || true - fi - else - if test -n "$compare_staged" - then - ls_list=$(git ls-tree HEAD "$name") - if test -n "$ls_list" - then - mkdir -p "$tmp/$left_dir/$(dirname "$name")" - git show HEAD:"$name" >"$tmp/$left_dir/$name" - fi - else - mkdir -p "$tmp/$left_dir/$(dirname "$name")" - git show :"$name" >"$tmp/$left_dir/$name" - fi - fi -done < "$tmp/filelist" - -LOCAL="$tmp/$left_dir" -REMOTE="$tmp/$right_dir" - -if test -n "$diff_tool" -then - export BASE - eval $diff_tool '"$LOCAL"' '"$REMOTE"' -else - run_merge_tool "$merge_tool" false -fi - -# Copy files back to the working dir, if requested -if test -n "$copy_back" && test "$right_dir" = "working_tree" -then - cd "$start_dir" - git_top_dir=$(git rev-parse --show-toplevel) - find "$tmp/$right_dir" -type f | - while read file - do - cp "$file" "$git_top_dir/${file#$tmp/$right_dir/}" - done -fi diff --git a/contrib/examples/builtin-fetch--tool.c b/contrib/examples/builtin-fetch--tool.c index 8bc8c7533a..a3eb19de04 100644 --- a/contrib/examples/builtin-fetch--tool.c +++ b/contrib/examples/builtin-fetch--tool.c @@ -31,7 +31,8 @@ static int update_ref_env(const char *action, rla = "(reflog update)"; if (snprintf(msg, sizeof(msg), "%s: %s", rla, action) >= sizeof(msg)) warning("reflog message too long: %.*s...", 50, msg); - return update_ref(msg, refname, sha1, oldval, 0, QUIET_ON_ERR); + return update_ref(msg, refname, sha1, oldval, 0, + UPDATE_REFS_QUIET_ON_ERR); } static int update_local_ref(const char *name, @@ -515,7 +516,7 @@ int cmd_fetch__tool(int argc, const char **argv, const char *prefix) if (argc != 8) return error("append-fetch-head takes 6 args"); - filename = git_path("FETCH_HEAD"); + filename = git_path_fetch_head(); fp = fopen(filename, "a"); if (!fp) return error("cannot open %s: %s", filename, strerror(errno)); @@ -533,7 +534,7 @@ int cmd_fetch__tool(int argc, const char **argv, const char *prefix) if (argc != 5) return error("fetch-native-store takes 3 args"); - filename = git_path("FETCH_HEAD"); + filename = git_path_fetch_head(); fp = fopen(filename, "a"); if (!fp) return error("cannot open %s: %s", filename, strerror(errno)); diff --git a/contrib/examples/git-am.sh b/contrib/examples/git-am.sh new file mode 100755 index 0000000000..3b77028123 --- /dev/null +++ b/contrib/examples/git-am.sh @@ -0,0 +1,975 @@ +#!/bin/sh +# +# Copyright (c) 2005, 2006 Junio C Hamano + +SUBDIRECTORY_OK=Yes +OPTIONS_KEEPDASHDASH= +OPTIONS_STUCKLONG=t +OPTIONS_SPEC="\ +git am [options] [(<mbox>|<Maildir>)...] +git am [options] (--continue | --skip | --abort) +-- +i,interactive run interactively +b,binary* (historical option -- no-op) +3,3way allow fall back on 3way merging if needed +q,quiet be quiet +s,signoff add a Signed-off-by line to the commit message +u,utf8 recode into utf8 (default) +k,keep pass -k flag to git-mailinfo +keep-non-patch pass -b flag to git-mailinfo +m,message-id pass -m flag to git-mailinfo +keep-cr pass --keep-cr flag to git-mailsplit for mbox format +no-keep-cr do not pass --keep-cr flag to git-mailsplit independent of am.keepcr +c,scissors strip everything before a scissors line +whitespace= pass it through git-apply +ignore-space-change pass it through git-apply +ignore-whitespace pass it through git-apply +directory= pass it through git-apply +exclude= pass it through git-apply +include= pass it through git-apply +C= pass it through git-apply +p= pass it through git-apply +patch-format= format the patch(es) are in +reject pass it through git-apply +resolvemsg= override error message when patch failure occurs +continue continue applying patches after resolving a conflict +r,resolved synonyms for --continue +skip skip the current patch +abort restore the original branch and abort the patching operation. +committer-date-is-author-date lie about committer date +ignore-date use current timestamp for author date +rerere-autoupdate update the index with reused conflict resolution if possible +S,gpg-sign? GPG-sign commits +rebasing* (internal use for git-rebase)" + +. git-sh-setup +. git-sh-i18n +prefix=$(git rev-parse --show-prefix) +set_reflog_action am +require_work_tree +cd_to_toplevel + +git var GIT_COMMITTER_IDENT >/dev/null || + die "$(gettext "You need to set your committer info first")" + +if git rev-parse --verify -q HEAD >/dev/null +then + HAS_HEAD=yes +else + HAS_HEAD= +fi + +cmdline="git am" +if test '' != "$interactive" +then + cmdline="$cmdline -i" +fi +if test '' != "$threeway" +then + cmdline="$cmdline -3" +fi + +empty_tree=4b825dc642cb6eb9a060e54bf8d69288fbee4904 + +sq () { + git rev-parse --sq-quote "$@" +} + +stop_here () { + echo "$1" >"$dotest/next" + git rev-parse --verify -q HEAD >"$dotest/abort-safety" + exit 1 +} + +safe_to_abort () { + if test -f "$dotest/dirtyindex" + then + return 1 + fi + + if ! test -f "$dotest/abort-safety" + then + return 0 + fi + + abort_safety=$(cat "$dotest/abort-safety") + if test "z$(git rev-parse --verify -q HEAD)" = "z$abort_safety" + then + return 0 + fi + gettextln "You seem to have moved HEAD since the last 'am' failure. +Not rewinding to ORIG_HEAD" >&2 + return 1 +} + +stop_here_user_resolve () { + if [ -n "$resolvemsg" ]; then + printf '%s\n' "$resolvemsg" + stop_here $1 + fi + eval_gettextln "When you have resolved this problem, run \"\$cmdline --continue\". +If you prefer to skip this patch, run \"\$cmdline --skip\" instead. +To restore the original branch and stop patching, run \"\$cmdline --abort\"." + + stop_here $1 +} + +go_next () { + rm -f "$dotest/$msgnum" "$dotest/msg" "$dotest/msg-clean" \ + "$dotest/patch" "$dotest/info" + echo "$next" >"$dotest/next" + this=$next +} + +cannot_fallback () { + echo "$1" + gettextln "Cannot fall back to three-way merge." + exit 1 +} + +fall_back_3way () { + O_OBJECT=$(cd "$GIT_OBJECT_DIRECTORY" && pwd) + + rm -fr "$dotest"/patch-merge-* + mkdir "$dotest/patch-merge-tmp-dir" + + # First see if the patch records the index info that we can use. + cmd="git apply $git_apply_opt --build-fake-ancestor" && + cmd="$cmd "'"$dotest/patch-merge-tmp-index" "$dotest/patch"' && + eval "$cmd" && + GIT_INDEX_FILE="$dotest/patch-merge-tmp-index" \ + git write-tree >"$dotest/patch-merge-base+" || + cannot_fallback "$(gettext "Repository lacks necessary blobs to fall back on 3-way merge.")" + + say "$(gettext "Using index info to reconstruct a base tree...")" + + cmd='GIT_INDEX_FILE="$dotest/patch-merge-tmp-index"' + + if test -z "$GIT_QUIET" + then + eval "$cmd git diff-index --cached --diff-filter=AM --name-status HEAD" + fi + + cmd="$cmd git apply --cached $git_apply_opt"' <"$dotest/patch"' + if eval "$cmd" + then + mv "$dotest/patch-merge-base+" "$dotest/patch-merge-base" + mv "$dotest/patch-merge-tmp-index" "$dotest/patch-merge-index" + else + cannot_fallback "$(gettext "Did you hand edit your patch? +It does not apply to blobs recorded in its index.")" + fi + + test -f "$dotest/patch-merge-index" && + his_tree=$(GIT_INDEX_FILE="$dotest/patch-merge-index" git write-tree) && + orig_tree=$(cat "$dotest/patch-merge-base") && + rm -fr "$dotest"/patch-merge-* || exit 1 + + say "$(gettext "Falling back to patching base and 3-way merge...")" + + # This is not so wrong. Depending on which base we picked, + # orig_tree may be wildly different from ours, but his_tree + # has the same set of wildly different changes in parts the + # patch did not touch, so recursive ends up canceling them, + # saying that we reverted all those changes. + + eval GITHEAD_$his_tree='"$FIRSTLINE"' + export GITHEAD_$his_tree + if test -n "$GIT_QUIET" + then + GIT_MERGE_VERBOSITY=0 && export GIT_MERGE_VERBOSITY + fi + our_tree=$(git rev-parse --verify -q HEAD || echo $empty_tree) + git-merge-recursive $orig_tree -- $our_tree $his_tree || { + git rerere $allow_rerere_autoupdate + die "$(gettext "Failed to merge in the changes.")" + } + unset GITHEAD_$his_tree +} + +clean_abort () { + test $# = 0 || echo >&2 "$@" + rm -fr "$dotest" + exit 1 +} + +patch_format= + +check_patch_format () { + # early return if patch_format was set from the command line + if test -n "$patch_format" + then + return 0 + fi + + # we default to mbox format if input is from stdin and for + # directories + if test $# = 0 || test "x$1" = "x-" || test -d "$1" + then + patch_format=mbox + return 0 + fi + + # otherwise, check the first few non-blank lines of the first + # patch to try to detect its format + { + # Start from first line containing non-whitespace + l1= + while test -z "$l1" + do + read l1 || break + done + read l2 + read l3 + case "$l1" in + "From "* | "From: "*) + patch_format=mbox + ;; + '# This series applies on GIT commit'*) + patch_format=stgit-series + ;; + "# HG changeset patch") + patch_format=hg + ;; + *) + # if the second line is empty and the third is + # a From, Author or Date entry, this is very + # likely an StGIT patch + case "$l2,$l3" in + ,"From: "* | ,"Author: "* | ,"Date: "*) + patch_format=stgit + ;; + *) + ;; + esac + ;; + esac + if test -z "$patch_format" && + test -n "$l1" && + test -n "$l2" && + test -n "$l3" + then + # This begins with three non-empty lines. Is this a + # piece of e-mail a-la RFC2822? Grab all the headers, + # discarding the indented remainder of folded lines, + # and see if it looks like that they all begin with the + # header field names... + tr -d '\015' <"$1" | + sed -n -e '/^$/q' -e '/^[ ]/d' -e p | + sane_egrep -v '^[!-9;-~]+:' >/dev/null || + patch_format=mbox + fi + } < "$1" || clean_abort +} + +split_patches () { + case "$patch_format" in + mbox) + if test t = "$keepcr" + then + keep_cr=--keep-cr + else + keep_cr= + fi + git mailsplit -d"$prec" -o"$dotest" -b $keep_cr -- "$@" > "$dotest/last" || + clean_abort + ;; + stgit-series) + if test $# -ne 1 + then + clean_abort "$(gettext "Only one StGIT patch series can be applied at once")" + fi + series_dir=$(dirname "$1") + series_file="$1" + shift + { + set x + while read filename + do + set "$@" "$series_dir/$filename" + done + # remove the safety x + shift + # remove the arg coming from the first-line comment + shift + } < "$series_file" || clean_abort + # set the patch format appropriately + patch_format=stgit + # now handle the actual StGIT patches + split_patches "$@" + ;; + stgit) + this=0 + test 0 -eq "$#" && set -- - + for stgit in "$@" + do + this=$(expr "$this" + 1) + msgnum=$(printf "%0${prec}d" $this) + # Perl version of StGIT parse_patch. The first nonemptyline + # not starting with Author, From or Date is the + # subject, and the body starts with the next nonempty + # line not starting with Author, From or Date + @@PERL@@ -ne 'BEGIN { $subject = 0 } + if ($subject > 1) { print ; } + elsif (/^\s+$/) { next ; } + elsif (/^Author:/) { s/Author/From/ ; print ;} + elsif (/^(From|Date)/) { print ; } + elsif ($subject) { + $subject = 2 ; + print "\n" ; + print ; + } else { + print "Subject: ", $_ ; + $subject = 1; + } + ' -- "$stgit" >"$dotest/$msgnum" || clean_abort + done + echo "$this" > "$dotest/last" + this= + msgnum= + ;; + hg) + this=0 + test 0 -eq "$#" && set -- - + for hg in "$@" + do + this=$(( $this + 1 )) + msgnum=$(printf "%0${prec}d" $this) + # hg stores changeset metadata in #-commented lines preceding + # the commit message and diff(s). The only metadata we care about + # are the User and Date (Node ID and Parent are hashes which are + # only relevant to the hg repository and thus not useful to us) + # Since we cannot guarantee that the commit message is in + # git-friendly format, we put no Subject: line and just consume + # all of the message as the body + LANG=C LC_ALL=C @@PERL@@ -M'POSIX qw(strftime)' -ne 'BEGIN { $subject = 0 } + if ($subject) { print ; } + elsif (/^\# User /) { s/\# User/From:/ ; print ; } + elsif (/^\# Date /) { + my ($hashsign, $str, $time, $tz) = split ; + $tz_str = sprintf "%+05d", (0-$tz)/36; + print "Date: " . + strftime("%a, %d %b %Y %H:%M:%S ", + gmtime($time-$tz)) + . "$tz_str\n"; + } elsif (/^\# /) { next ; } + else { + print "\n", $_ ; + $subject = 1; + } + ' -- "$hg" >"$dotest/$msgnum" || clean_abort + done + echo "$this" >"$dotest/last" + this= + msgnum= + ;; + *) + if test -n "$patch_format" + then + clean_abort "$(eval_gettext "Patch format \$patch_format is not supported.")" + else + clean_abort "$(gettext "Patch format detection failed.")" + fi + ;; + esac +} + +prec=4 +dotest="$GIT_DIR/rebase-apply" +sign= utf8=t keep= keepcr= skip= interactive= resolved= rebasing= abort= +messageid= resolvemsg= resume= scissors= no_inbody_headers= +git_apply_opt= +committer_date_is_author_date= +ignore_date= +allow_rerere_autoupdate= +gpg_sign_opt= +threeway= + +if test "$(git config --bool --get am.messageid)" = true +then + messageid=t +fi + +if test "$(git config --bool --get am.keepcr)" = true +then + keepcr=t +fi + +while test $# != 0 +do + case "$1" in + -i|--interactive) + interactive=t ;; + -b|--binary) + gettextln >&2 "The -b/--binary option has been a no-op for long time, and +it will be removed. Please do not use it anymore." + ;; + -3|--3way) + threeway=t ;; + -s|--signoff) + sign=t ;; + -u|--utf8) + utf8=t ;; # this is now default + --no-utf8) + utf8= ;; + -m|--message-id) + messageid=t ;; + --no-message-id) + messageid=f ;; + -k|--keep) + keep=t ;; + --keep-non-patch) + keep=b ;; + -c|--scissors) + scissors=t ;; + --no-scissors) + scissors=f ;; + -r|--resolved|--continue) + resolved=t ;; + --skip) + skip=t ;; + --abort) + abort=t ;; + --rebasing) + rebasing=t threeway=t ;; + --resolvemsg=*) + resolvemsg="${1#--resolvemsg=}" ;; + --whitespace=*|--directory=*|--exclude=*|--include=*) + git_apply_opt="$git_apply_opt $(sq "$1")" ;; + -C*|-p*) + git_apply_opt="$git_apply_opt $(sq "$1")" ;; + --patch-format=*) + patch_format="${1#--patch-format=}" ;; + --reject|--ignore-whitespace|--ignore-space-change) + git_apply_opt="$git_apply_opt $1" ;; + --committer-date-is-author-date) + committer_date_is_author_date=t ;; + --ignore-date) + ignore_date=t ;; + --rerere-autoupdate|--no-rerere-autoupdate) + allow_rerere_autoupdate="$1" ;; + -q|--quiet) + GIT_QUIET=t ;; + --keep-cr) + keepcr=t ;; + --no-keep-cr) + keepcr=f ;; + --gpg-sign) + gpg_sign_opt=-S ;; + --gpg-sign=*) + gpg_sign_opt="-S${1#--gpg-sign=}" ;; + --) + shift; break ;; + *) + usage ;; + esac + shift +done + +# If the dotest directory exists, but we have finished applying all the +# patches in them, clear it out. +if test -d "$dotest" && + test -f "$dotest/last" && + test -f "$dotest/next" && + last=$(cat "$dotest/last") && + next=$(cat "$dotest/next") && + test $# != 0 && + test "$next" -gt "$last" +then + rm -fr "$dotest" +fi + +if test -d "$dotest" && test -f "$dotest/last" && test -f "$dotest/next" +then + case "$#,$skip$resolved$abort" in + 0,*t*) + # Explicit resume command and we do not have file, so + # we are happy. + : ;; + 0,) + # No file input but without resume parameters; catch + # user error to feed us a patch from standard input + # when there is already $dotest. This is somewhat + # unreliable -- stdin could be /dev/null for example + # and the caller did not intend to feed us a patch but + # wanted to continue unattended. + test -t 0 + ;; + *) + false + ;; + esac || + die "$(eval_gettext "previous rebase directory \$dotest still exists but mbox given.")" + resume=yes + + case "$skip,$abort" in + t,t) + die "$(gettext "Please make up your mind. --skip or --abort?")" + ;; + t,) + git rerere clear + head_tree=$(git rev-parse --verify -q HEAD || echo $empty_tree) && + git read-tree --reset -u $head_tree $head_tree && + index_tree=$(git write-tree) && + git read-tree -m -u $index_tree $head_tree + git read-tree $head_tree + ;; + ,t) + if test -f "$dotest/rebasing" + then + exec git rebase --abort + fi + git rerere clear + if safe_to_abort + then + head_tree=$(git rev-parse --verify -q HEAD || echo $empty_tree) && + git read-tree --reset -u $head_tree $head_tree && + index_tree=$(git write-tree) && + orig_head=$(git rev-parse --verify -q ORIG_HEAD || echo $empty_tree) && + git read-tree -m -u $index_tree $orig_head + if git rev-parse --verify -q ORIG_HEAD >/dev/null 2>&1 + then + git reset ORIG_HEAD + else + git read-tree $empty_tree + curr_branch=$(git symbolic-ref HEAD 2>/dev/null) && + git update-ref -d $curr_branch + fi + fi + rm -fr "$dotest" + exit ;; + esac + rm -f "$dotest/dirtyindex" +else + # Possible stray $dotest directory in the independent-run + # case; in the --rebasing case, it is upto the caller + # (git-rebase--am) to take care of stray directories. + if test -d "$dotest" && test -z "$rebasing" + then + case "$skip,$resolved,$abort" in + ,,t) + rm -fr "$dotest" + exit 0 + ;; + *) + die "$(eval_gettext "Stray \$dotest directory found. +Use \"git am --abort\" to remove it.")" + ;; + esac + fi + + # Make sure we are not given --skip, --continue, or --abort + test "$skip$resolved$abort" = "" || + die "$(gettext "Resolve operation not in progress, we are not resuming.")" + + # Start afresh. + mkdir -p "$dotest" || exit + + if test -n "$prefix" && test $# != 0 + then + first=t + for arg + do + test -n "$first" && { + set x + first= + } + if is_absolute_path "$arg" + then + set "$@" "$arg" + else + set "$@" "$prefix$arg" + fi + done + shift + fi + + check_patch_format "$@" + + split_patches "$@" + + # -i can and must be given when resuming; everything + # else is kept + echo " $git_apply_opt" >"$dotest/apply-opt" + echo "$threeway" >"$dotest/threeway" + echo "$sign" >"$dotest/sign" + echo "$utf8" >"$dotest/utf8" + echo "$keep" >"$dotest/keep" + echo "$messageid" >"$dotest/messageid" + echo "$scissors" >"$dotest/scissors" + echo "$no_inbody_headers" >"$dotest/no_inbody_headers" + echo "$GIT_QUIET" >"$dotest/quiet" + echo 1 >"$dotest/next" + if test -n "$rebasing" + then + : >"$dotest/rebasing" + else + : >"$dotest/applying" + if test -n "$HAS_HEAD" + then + git update-ref ORIG_HEAD HEAD + else + git update-ref -d ORIG_HEAD >/dev/null 2>&1 + fi + fi +fi + +git update-index -q --refresh + +case "$resolved" in +'') + case "$HAS_HEAD" in + '') + files=$(git ls-files) ;; + ?*) + files=$(git diff-index --cached --name-only HEAD --) ;; + esac || exit + if test "$files" + then + test -n "$HAS_HEAD" && : >"$dotest/dirtyindex" + die "$(eval_gettext "Dirty index: cannot apply patches (dirty: \$files)")" + fi +esac + +# Now, decide what command line options we will give to the git +# commands we invoke, based on the result of parsing command line +# options and previous invocation state stored in $dotest/ files. + +if test "$(cat "$dotest/utf8")" = t +then + utf8=-u +else + utf8=-n +fi +keep=$(cat "$dotest/keep") +case "$keep" in +t) + keep=-k ;; +b) + keep=-b ;; +*) + keep= ;; +esac +case "$(cat "$dotest/messageid")" in +t) + messageid=-m ;; +f) + messageid= ;; +esac +case "$(cat "$dotest/scissors")" in +t) + scissors=--scissors ;; +f) + scissors=--no-scissors ;; +esac +if test "$(cat "$dotest/no_inbody_headers")" = t +then + no_inbody_headers=--no-inbody-headers +else + no_inbody_headers= +fi +if test "$(cat "$dotest/quiet")" = t +then + GIT_QUIET=t +fi +if test "$(cat "$dotest/threeway")" = t +then + threeway=t +fi +git_apply_opt=$(cat "$dotest/apply-opt") +if test "$(cat "$dotest/sign")" = t +then + SIGNOFF=$(git var GIT_COMMITTER_IDENT | sed -e ' + s/>.*/>/ + s/^/Signed-off-by: /' + ) +else + SIGNOFF= +fi + +last=$(cat "$dotest/last") +this=$(cat "$dotest/next") +if test "$skip" = t +then + this=$(expr "$this" + 1) + resume= +fi + +while test "$this" -le "$last" +do + msgnum=$(printf "%0${prec}d" $this) + next=$(expr "$this" + 1) + test -f "$dotest/$msgnum" || { + resume= + go_next + continue + } + + # If we are not resuming, parse and extract the patch information + # into separate files: + # - info records the authorship and title + # - msg is the rest of commit log message + # - patch is the patch body. + # + # When we are resuming, these files are either already prepared + # by the user, or the user can tell us to do so by --continue flag. + case "$resume" in + '') + if test -f "$dotest/rebasing" + then + commit=$(sed -e 's/^From \([0-9a-f]*\) .*/\1/' \ + -e q "$dotest/$msgnum") && + test "$(git cat-file -t "$commit")" = commit || + stop_here $this + git cat-file commit "$commit" | + sed -e '1,/^$/d' >"$dotest/msg-clean" + echo "$commit" >"$dotest/original-commit" + get_author_ident_from_commit "$commit" >"$dotest/author-script" + git diff-tree --root --binary --full-index "$commit" >"$dotest/patch" + else + git mailinfo $keep $no_inbody_headers $messageid $scissors $utf8 "$dotest/msg" "$dotest/patch" \ + <"$dotest/$msgnum" >"$dotest/info" || + stop_here $this + + # skip pine's internal folder data + sane_grep '^Author: Mail System Internal Data$' \ + <"$dotest"/info >/dev/null && + go_next && continue + + test -s "$dotest/patch" || { + eval_gettextln "Patch is empty. Was it split wrong? +If you would prefer to skip this patch, instead run \"\$cmdline --skip\". +To restore the original branch and stop patching run \"\$cmdline --abort\"." + stop_here $this + } + rm -f "$dotest/original-commit" "$dotest/author-script" + { + sed -n '/^Subject/ s/Subject: //p' "$dotest/info" + echo + cat "$dotest/msg" + } | + git stripspace > "$dotest/msg-clean" + fi + ;; + esac + + if test -f "$dotest/author-script" + then + eval $(cat "$dotest/author-script") + else + GIT_AUTHOR_NAME="$(sed -n '/^Author/ s/Author: //p' "$dotest/info")" + GIT_AUTHOR_EMAIL="$(sed -n '/^Email/ s/Email: //p' "$dotest/info")" + GIT_AUTHOR_DATE="$(sed -n '/^Date/ s/Date: //p' "$dotest/info")" + fi + + if test -z "$GIT_AUTHOR_EMAIL" + then + gettextln "Patch does not have a valid e-mail address." + stop_here $this + fi + + export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE + + case "$resume" in + '') + if test '' != "$SIGNOFF" + then + LAST_SIGNED_OFF_BY=$( + sed -ne '/^Signed-off-by: /p' \ + "$dotest/msg-clean" | + sed -ne '$p' + ) + ADD_SIGNOFF=$( + test "$LAST_SIGNED_OFF_BY" = "$SIGNOFF" || { + test '' = "$LAST_SIGNED_OFF_BY" && echo + echo "$SIGNOFF" + }) + else + ADD_SIGNOFF= + fi + { + if test -s "$dotest/msg-clean" + then + cat "$dotest/msg-clean" + fi + if test '' != "$ADD_SIGNOFF" + then + echo "$ADD_SIGNOFF" + fi + } >"$dotest/final-commit" + ;; + *) + case "$resolved$interactive" in + tt) + # This is used only for interactive view option. + git diff-index -p --cached HEAD -- >"$dotest/patch" + ;; + esac + esac + + resume= + if test "$interactive" = t + then + test -t 0 || + die "$(gettext "cannot be interactive without stdin connected to a terminal.")" + action=again + while test "$action" = again + do + gettextln "Commit Body is:" + echo "--------------------------" + cat "$dotest/final-commit" + echo "--------------------------" + # TRANSLATORS: Make sure to include [y], [n], [e], [v] and [a] + # in your translation. The program will only accept English + # input at this point. + gettext "Apply? [y]es/[n]o/[e]dit/[v]iew patch/[a]ccept all " + read reply + case "$reply" in + [yY]*) action=yes ;; + [aA]*) action=yes interactive= ;; + [nN]*) action=skip ;; + [eE]*) git_editor "$dotest/final-commit" + action=again ;; + [vV]*) action=again + git_pager "$dotest/patch" ;; + *) action=again ;; + esac + done + else + action=yes + fi + + if test $action = skip + then + go_next + continue + fi + + hook="$(git rev-parse --git-path hooks/applypatch-msg)" + if test -x "$hook" + then + "$hook" "$dotest/final-commit" || stop_here $this + fi + + if test -f "$dotest/final-commit" + then + FIRSTLINE=$(sed 1q "$dotest/final-commit") + else + FIRSTLINE="" + fi + + say "$(eval_gettext "Applying: \$FIRSTLINE")" + + case "$resolved" in + '') + # When we are allowed to fall back to 3-way later, don't give + # false errors during the initial attempt. + squelch= + if test "$threeway" = t + then + squelch='>/dev/null 2>&1 ' + fi + eval "git apply $squelch$git_apply_opt"' --index "$dotest/patch"' + apply_status=$? + ;; + t) + # Resolved means the user did all the hard work, and + # we do not have to do any patch application. Just + # trust what the user has in the index file and the + # working tree. + resolved= + git diff-index --quiet --cached HEAD -- && { + gettextln "No changes - did you forget to use 'git add'? +If there is nothing left to stage, chances are that something else +already introduced the same changes; you might want to skip this patch." + stop_here_user_resolve $this + } + unmerged=$(git ls-files -u) + if test -n "$unmerged" + then + gettextln "You still have unmerged paths in your index +did you forget to use 'git add'?" + stop_here_user_resolve $this + fi + apply_status=0 + git rerere + ;; + esac + + if test $apply_status != 0 && test "$threeway" = t + then + if (fall_back_3way) + then + # Applying the patch to an earlier tree and merging the + # result may have produced the same tree as ours. + git diff-index --quiet --cached HEAD -- && { + say "$(gettext "No changes -- Patch already applied.")" + go_next + continue + } + # clear apply_status -- we have successfully merged. + apply_status=0 + fi + fi + if test $apply_status != 0 + then + eval_gettextln 'Patch failed at $msgnum $FIRSTLINE' + if test "$(git config --bool advice.amworkdir)" != false + then + eval_gettextln 'The copy of the patch that failed is found in: + $dotest/patch' + fi + stop_here_user_resolve $this + fi + + hook="$(git rev-parse --git-path hooks/pre-applypatch)" + if test -x "$hook" + then + "$hook" || stop_here $this + fi + + tree=$(git write-tree) && + commit=$( + if test -n "$ignore_date" + then + GIT_AUTHOR_DATE= + fi + parent=$(git rev-parse --verify -q HEAD) || + say >&2 "$(gettext "applying to an empty history")" + + if test -n "$committer_date_is_author_date" + then + GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" + export GIT_COMMITTER_DATE + fi && + git commit-tree ${parent:+-p} $parent ${gpg_sign_opt:+"$gpg_sign_opt"} $tree \ + <"$dotest/final-commit" + ) && + git update-ref -m "$GIT_REFLOG_ACTION: $FIRSTLINE" HEAD $commit $parent || + stop_here $this + + if test -f "$dotest/original-commit"; then + echo "$(cat "$dotest/original-commit") $commit" >> "$dotest/rewritten" + fi + + hook="$(git rev-parse --git-path hooks/post-applypatch)" + test -x "$hook" && "$hook" + + go_next +done + +if test -s "$dotest"/rewritten; then + git notes copy --for-rewrite=rebase < "$dotest"/rewritten + hook="$(git rev-parse --git-path hooks/post-rewrite)" + if test -x "$hook"; then + "$hook" rebase < "$dotest"/rewritten + fi +fi + +# If am was called with --rebasing (from git-rebase--am), it's up to +# the caller to take care of housekeeping. +if ! test -f "$dotest/rebasing" +then + rm -fr "$dotest" + git gc --auto +fi diff --git a/contrib/examples/git-checkout.sh b/contrib/examples/git-checkout.sh index 1a7689a48f..683cae7c3f 100755 --- a/contrib/examples/git-checkout.sh +++ b/contrib/examples/git-checkout.sh @@ -168,7 +168,7 @@ cd_to_toplevel # branch. However, if "git checkout HEAD" detaches the HEAD # from the current branch, even though that may be logically # correct, it feels somewhat funny. More importantly, we do not -# want "git checkout" nor "git checkout -f" to detach HEAD. +# want "git checkout" or "git checkout -f" to detach HEAD. detached= detach_warn= @@ -222,7 +222,7 @@ else # Match the index to the working tree, and do a three-way. git diff-files --name-only | git update-index --remove --stdin && - work=`git write-tree` && + work=$(git write-tree) && git read-tree $v --reset -u $new || exit eval GITHEAD_$new='${new_name:-${branch:-$new}}' && @@ -233,7 +233,7 @@ else # Do not register the cleanly merged paths in the index yet. # this is not a real merge before committing, but just carrying # the working tree changes along. - unmerged=`git ls-files -u` + unmerged=$(git ls-files -u) git read-tree $v --reset $new case "$unmerged" in '') ;; @@ -269,7 +269,7 @@ if [ "$?" -eq 0 ]; then fi if test -n "$branch" then - old_branch_name=`expr "z$oldbranch" : 'zrefs/heads/\(.*\)'` + old_branch_name=$(expr "z$oldbranch" : 'zrefs/heads/\(.*\)') GIT_DIR="$GIT_DIR" git symbolic-ref -m "checkout: moving from ${old_branch_name:-$old} to $branch" HEAD "refs/heads/$branch" if test -n "$quiet" then @@ -282,7 +282,7 @@ if [ "$?" -eq 0 ]; then fi elif test -n "$detached" then - old_branch_name=`expr "z$oldbranch" : 'zrefs/heads/\(.*\)'` + old_branch_name=$(expr "z$oldbranch" : 'zrefs/heads/\(.*\)') git update-ref --no-deref -m "checkout: moving from ${old_branch_name:-$old} to $arg" HEAD "$detached" || die "Cannot detach HEAD" if test -n "$detach_warn" diff --git a/contrib/examples/git-clone.sh b/contrib/examples/git-clone.sh index 547228e13c..08cf246bbb 100755 --- a/contrib/examples/git-clone.sh +++ b/contrib/examples/git-clone.sh @@ -40,7 +40,7 @@ eval "$(echo "$OPTIONS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?) get_repo_base() { ( - cd "`/bin/pwd`" && + cd "$(/bin/pwd)" && cd "$1" || cd "$1.git" && { cd .git @@ -50,7 +50,7 @@ get_repo_base() { } if [ -n "$GIT_SSL_NO_VERIFY" -o \ - "`git config --bool http.sslVerify`" = false ]; then + "$(git config --bool http.sslVerify)" = false ]; then curl_extra_args="-k" fi @@ -70,7 +70,7 @@ clone_dumb_http () { clone_tmp="$GIT_DIR/clone-tmp" && mkdir -p "$clone_tmp" || exit 1 if [ -n "$GIT_CURL_FTP_NO_EPSV" -o \ - "`git config --bool http.noEPSV`" = true ]; then + "$(git config --bool http.noEPSV)" = true ]; then curl_extra_args="${curl_extra_args} --disable-epsv" fi http_fetch "$1/info/refs" "$clone_tmp/refs" || @@ -79,7 +79,7 @@ Perhaps git-update-server-info needs to be run there?" test "z$quiet" = z && v=-v || v= while read sha1 refname do - name=`expr "z$refname" : 'zrefs/\(.*\)'` && + name=$(expr "z$refname" : 'zrefs/\(.*\)') && case "$name" in *^*) continue;; esac @@ -88,7 +88,7 @@ Perhaps git-update-server-info needs to be run there?" *) continue ;; esac if test -n "$use_separate_remote" && - branch_name=`expr "z$name" : 'zheads/\(.*\)'` + branch_name=$(expr "z$name" : 'zheads/\(.*\)') then tname="remotes/$origin/$branch_name" else @@ -100,7 +100,7 @@ Perhaps git-update-server-info needs to be run there?" http_fetch "$1/HEAD" "$GIT_DIR/REMOTE_HEAD" || rm -f "$GIT_DIR/REMOTE_HEAD" if test -f "$GIT_DIR/REMOTE_HEAD"; then - head_sha1=`cat "$GIT_DIR/REMOTE_HEAD"` + head_sha1=$(cat "$GIT_DIR/REMOTE_HEAD") case "$head_sha1" in 'ref: refs/'*) ;; @@ -444,15 +444,15 @@ then # a non-bare repository is always in separate-remote layout remote_top="refs/remotes/$origin" head_sha1= - test ! -r "$GIT_DIR/REMOTE_HEAD" || head_sha1=`cat "$GIT_DIR/REMOTE_HEAD"` + test ! -r "$GIT_DIR/REMOTE_HEAD" || head_sha1=$(cat "$GIT_DIR/REMOTE_HEAD") case "$head_sha1" in 'ref: refs/'*) # Uh-oh, the remote told us (http transport done against # new style repository with a symref HEAD). # Ideally we should skip the guesswork but for now # opt for minimum change. - head_sha1=`expr "z$head_sha1" : 'zref: refs/heads/\(.*\)'` - head_sha1=`cat "$GIT_DIR/$remote_top/$head_sha1"` + head_sha1=$(expr "z$head_sha1" : 'zref: refs/heads/\(.*\)') + head_sha1=$(cat "$GIT_DIR/$remote_top/$head_sha1") ;; esac @@ -467,7 +467,7 @@ then while read name do test t = $done && continue - branch_tip=`cat "$GIT_DIR/$remote_top/$name"` + branch_tip=$(cat "$GIT_DIR/$remote_top/$name") if test "$head_sha1" = "$branch_tip" then echo "$name" @@ -516,7 +516,7 @@ then case "$no_checkout" in '') - test "z$quiet" = z -a "z$no_progress" = z && v=-v || v= + test "z$quiet" = z && test "z$no_progress" = z && v=-v || v= git read-tree -m -u $v HEAD HEAD esac fi diff --git a/contrib/examples/git-commit.sh b/contrib/examples/git-commit.sh index 23ffb028d1..934505bab9 100755 --- a/contrib/examples/git-commit.sh +++ b/contrib/examples/git-commit.sh @@ -51,7 +51,7 @@ run_status () { export GIT_INDEX_FILE fi - if test "$status_only" = "t" -o "$use_status_color" = "t"; then + if test "$status_only" = "t" || test "$use_status_color" = "t"; then color= else color=--nocolor @@ -91,7 +91,7 @@ signoff= force_author= only_include_assumed= untracked_files= -templatefile="`git config commit.template`" +templatefile="$(git config commit.template)" while test $# != 0 do case "$1" in @@ -280,7 +280,7 @@ case "$#,$also,$only,$amend" in 0,,,*) ;; *,,,*) - only_include_assumed="# Explicit paths specified without -i nor -o; assuming --only paths..." + only_include_assumed="# Explicit paths specified without -i or -o; assuming --only paths..." also= ;; esac @@ -296,7 +296,7 @@ t,,,[1-9]*) die "No paths with -i does not make sense." ;; esac -if test ! -z "$templatefile" -a -z "$log_given" +if test ! -z "$templatefile" && test -z "$log_given" then if test ! -f "$templatefile" then @@ -350,7 +350,7 @@ t,) TMP_INDEX="$GIT_DIR/tmp-index$$" W= test -z "$initial_commit" && W=--with-tree=HEAD - commit_only=`git ls-files --error-unmatch $W -- "$@"` || exit + commit_only=$(git ls-files --error-unmatch $W -- "$@") || exit # Build a temporary index and update the real index # the same way. @@ -475,8 +475,8 @@ then fi if test '' != "$force_author" then - GIT_AUTHOR_NAME=`expr "z$force_author" : 'z\(.*[^ ]\) *<.*'` && - GIT_AUTHOR_EMAIL=`expr "z$force_author" : '.*\(<.*\)'` && + GIT_AUTHOR_NAME=$(expr "z$force_author" : 'z\(.*[^ ]\) *<.*') && + GIT_AUTHOR_EMAIL=$(expr "z$force_author" : '.*\(<.*\)') && test '' != "$GIT_AUTHOR_NAME" && test '' != "$GIT_AUTHOR_EMAIL" || die "malformed --author parameter" @@ -489,7 +489,7 @@ then rloga='commit' if [ -f "$GIT_DIR/MERGE_HEAD" ]; then rloga='commit (merge)' - PARENTS="-p HEAD "`sed -e 's/^/-p /' "$GIT_DIR/MERGE_HEAD"` + PARENTS="-p HEAD "$(sed -e 's/^/-p /' "$GIT_DIR/MERGE_HEAD") elif test -n "$amend"; then rloga='commit (amend)' PARENTS=$(git cat-file commit HEAD | diff --git a/contrib/examples/git-fetch.sh b/contrib/examples/git-fetch.sh index a314273bd5..554070909c 100755 --- a/contrib/examples/git-fetch.sh +++ b/contrib/examples/git-fetch.sh @@ -67,7 +67,7 @@ do keep='-k -k' ;; --depth=*) - shallow_depth="--depth=`expr "z$1" : 'z-[^=]*=\(.*\)'`" + shallow_depth="--depth=$(expr "z$1" : 'z-[^=]*=\(.*\)')" ;; --depth) shift @@ -262,12 +262,12 @@ fetch_per_ref () { http://* | https://* | ftp://*) test -n "$shallow_depth" && die "shallow clone with http not supported" - proto=`expr "$remote" : '\([^:]*\):'` + proto=$(expr "$remote" : '\([^:]*\):') if [ -n "$GIT_SSL_NO_VERIFY" ]; then curl_extra_args="-k" fi if [ -n "$GIT_CURL_FTP_NO_EPSV" -o \ - "`git config --bool http.noEPSV`" = true ]; then + "$(git config --bool http.noEPSV)" = true ]; then noepsv_opt="--disable-epsv" fi diff --git a/contrib/examples/git-log.sh b/contrib/examples/git-log.sh new file mode 100755 index 0000000000..c2ea71cf14 --- /dev/null +++ b/contrib/examples/git-log.sh @@ -0,0 +1,15 @@ +#!/bin/sh +# +# Copyright (c) 2005 Linus Torvalds +# + +USAGE='[--max-count=<n>] [<since>..<limit>] [--pretty=<format>] [git-rev-list options]' +SUBDIRECTORY_OK='Yes' +. git-sh-setup + +revs=$(git-rev-parse --revs-only --no-flags --default HEAD "$@") || exit +[ "$revs" ] || { + die "No HEAD ref" +} +git-rev-list --pretty $(git-rev-parse --default HEAD "$@") | +LESS=-S ${PAGER:-less} diff --git a/contrib/examples/git-ls-remote.sh b/contrib/examples/git-ls-remote.sh index fec70bbf88..2aa89a7df8 100755 --- a/contrib/examples/git-ls-remote.sh +++ b/contrib/examples/git-ls-remote.sh @@ -55,11 +55,11 @@ tmpdir=$tmp-d case "$peek_repo" in http://* | https://* | ftp://* ) if [ -n "$GIT_SSL_NO_VERIFY" -o \ - "`git config --bool http.sslVerify`" = false ]; then + "$(git config --bool http.sslVerify)" = false ]; then curl_extra_args="-k" fi if [ -n "$GIT_CURL_FTP_NO_EPSV" -o \ - "`git config --bool http.noEPSV`" = true ]; then + "$(git config --bool http.noEPSV)" = true ]; then curl_extra_args="${curl_extra_args} --disable-epsv" fi curl -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" || diff --git a/contrib/examples/git-merge.sh b/contrib/examples/git-merge.sh index 7b922c3948..52f2aafb9d 100755 --- a/contrib/examples/git-merge.sh +++ b/contrib/examples/git-merge.sh @@ -161,7 +161,7 @@ merge_name () { return fi fi - if test "$remote" = "FETCH_HEAD" -a -r "$GIT_DIR/FETCH_HEAD" + if test "$remote" = "FETCH_HEAD" && test -r "$GIT_DIR/FETCH_HEAD" then sed -e 's/ not-for-merge / /' -e 1q \ "$GIT_DIR/FETCH_HEAD" @@ -263,7 +263,7 @@ fi # This could be traditional "merge <msg> HEAD <commit>..." and the # way we can tell it is to see if the second token is HEAD, but some -# people might have misused the interface and used a committish that +# people might have misused the interface and used a commit-ish that # is the same as HEAD there instead. Traditional format never would # have "-m" so it is an additional safety measure to check for it. @@ -341,7 +341,7 @@ case "$use_strategies" in '') case "$#" in 1) - var="`git config --get pull.twohead`" + var="$(git config --get pull.twohead)" if test -n "$var" then use_strategies="$var" @@ -349,7 +349,7 @@ case "$use_strategies" in use_strategies="$default_twohead_strategies" fi ;; *) - var="`git config --get pull.octopus`" + var="$(git config --get pull.octopus)" if test -n "$var" then use_strategies="$var" @@ -527,7 +527,7 @@ do git diff-files --name-only git ls-files --unmerged } | wc -l` - if test $best_cnt -le 0 -o $cnt -le $best_cnt + if test $best_cnt -le 0 || test $cnt -le $best_cnt then best_strategy=$strategy best_cnt=$cnt diff --git a/contrib/examples/git-pull.sh b/contrib/examples/git-pull.sh new file mode 100755 index 0000000000..e8dc2e0e7d --- /dev/null +++ b/contrib/examples/git-pull.sh @@ -0,0 +1,381 @@ +#!/bin/sh +# +# Copyright (c) 2005 Junio C Hamano +# +# Fetch one or more remote refs and merge it/them into the current HEAD. + +SUBDIRECTORY_OK=Yes +OPTIONS_KEEPDASHDASH= +OPTIONS_STUCKLONG=Yes +OPTIONS_SPEC="\ +git pull [options] [<repository> [<refspec>...]] + +Fetch one or more remote refs and integrate it/them with the current HEAD. +-- +v,verbose be more verbose +q,quiet be more quiet +progress force progress reporting + + Options related to merging +r,rebase?false|true|preserve incorporate changes by rebasing rather than merging +n! do not show a diffstat at the end of the merge +stat show a diffstat at the end of the merge +summary (synonym to --stat) +log?n add (at most <n>) entries from shortlog to merge commit message +squash create a single commit instead of doing a merge +commit perform a commit if the merge succeeds (default) +e,edit edit message before committing +ff allow fast-forward +ff-only! abort if fast-forward is not possible +verify-signatures verify that the named commit has a valid GPG signature +s,strategy=strategy merge strategy to use +X,strategy-option=option option for selected merge strategy +S,gpg-sign?key-id GPG sign commit + + Options related to fetching +all fetch from all remotes +a,append append to .git/FETCH_HEAD instead of overwriting +upload-pack=path path to upload pack on remote end +f,force force overwrite of local branch +t,tags fetch all tags and associated objects +p,prune prune remote-tracking branches no longer on remote +recurse-submodules?on-demand control recursive fetching of submodules +dry-run dry run +k,keep keep downloaded pack +depth=depth deepen history of shallow clone +unshallow convert to a complete repository +update-shallow accept refs that update .git/shallow +refmap=refmap specify fetch refmap +" +test $# -gt 0 && args="$*" +. git-sh-setup +. git-sh-i18n +set_reflog_action "pull${args+ $args}" +require_work_tree_exists +cd_to_toplevel + + +die_conflict () { + git diff-index --cached --name-status -r --ignore-submodules HEAD -- + if [ $(git config --bool --get advice.resolveConflict || echo true) = "true" ]; then + die "$(gettext "Pull is not possible because you have unmerged files. +Please, fix them up in the work tree, and then use 'git add/rm <file>' +as appropriate to mark resolution and make a commit.")" + else + die "$(gettext "Pull is not possible because you have unmerged files.")" + fi +} + +die_merge () { + if [ $(git config --bool --get advice.resolveConflict || echo true) = "true" ]; then + die "$(gettext "You have not concluded your merge (MERGE_HEAD exists). +Please, commit your changes before you can merge.")" + else + die "$(gettext "You have not concluded your merge (MERGE_HEAD exists).")" + fi +} + +test -z "$(git ls-files -u)" || die_conflict +test -f "$GIT_DIR/MERGE_HEAD" && die_merge + +bool_or_string_config () { + git config --bool "$1" 2>/dev/null || git config "$1" +} + +strategy_args= diffstat= no_commit= squash= no_ff= ff_only= +log_arg= verbosity= progress= recurse_submodules= verify_signatures= +merge_args= edit= rebase_args= all= append= upload_pack= force= tags= prune= +keep= depth= unshallow= update_shallow= refmap= +curr_branch=$(git symbolic-ref -q HEAD) +curr_branch_short="${curr_branch#refs/heads/}" +rebase=$(bool_or_string_config branch.$curr_branch_short.rebase) +if test -z "$rebase" +then + rebase=$(bool_or_string_config pull.rebase) +fi + +# Setup default fast-forward options via `pull.ff` +pull_ff=$(bool_or_string_config pull.ff) +case "$pull_ff" in +true) + no_ff=--ff + ;; +false) + no_ff=--no-ff + ;; +only) + ff_only=--ff-only + ;; +esac + + +dry_run= +while : +do + case "$1" in + -q|--quiet) + verbosity="$verbosity -q" ;; + -v|--verbose) + verbosity="$verbosity -v" ;; + --progress) + progress=--progress ;; + --no-progress) + progress=--no-progress ;; + -n|--no-stat|--no-summary) + diffstat=--no-stat ;; + --stat|--summary) + diffstat=--stat ;; + --log|--log=*|--no-log) + log_arg="$1" ;; + --no-commit) + no_commit=--no-commit ;; + --commit) + no_commit=--commit ;; + -e|--edit) + edit=--edit ;; + --no-edit) + edit=--no-edit ;; + --squash) + squash=--squash ;; + --no-squash) + squash=--no-squash ;; + --ff) + no_ff=--ff ;; + --no-ff) + no_ff=--no-ff ;; + --ff-only) + ff_only=--ff-only ;; + -s*|--strategy=*) + strategy_args="$strategy_args $1" + ;; + -X*|--strategy-option=*) + merge_args="$merge_args $(git rev-parse --sq-quote "$1")" + ;; + -r*|--rebase=*) + rebase="${1#*=}" + ;; + --rebase) + rebase=true + ;; + --no-rebase) + rebase=false + ;; + --recurse-submodules) + recurse_submodules=--recurse-submodules + ;; + --recurse-submodules=*) + recurse_submodules="$1" + ;; + --no-recurse-submodules) + recurse_submodules=--no-recurse-submodules + ;; + --verify-signatures) + verify_signatures=--verify-signatures + ;; + --no-verify-signatures) + verify_signatures=--no-verify-signatures + ;; + --gpg-sign|-S) + gpg_sign_args=-S + ;; + --gpg-sign=*) + gpg_sign_args=$(git rev-parse --sq-quote "-S${1#--gpg-sign=}") + ;; + -S*) + gpg_sign_args=$(git rev-parse --sq-quote "$1") + ;; + --dry-run) + dry_run=--dry-run + ;; + --all|--no-all) + all=$1 ;; + -a|--append|--no-append) + append=$1 ;; + --upload-pack=*|--no-upload-pack) + upload_pack=$1 ;; + -f|--force|--no-force) + force="$force $1" ;; + -t|--tags|--no-tags) + tags=$1 ;; + -p|--prune|--no-prune) + prune=$1 ;; + -k|--keep|--no-keep) + keep=$1 ;; + --depth=*|--no-depth) + depth=$1 ;; + --unshallow|--no-unshallow) + unshallow=$1 ;; + --update-shallow|--no-update-shallow) + update_shallow=$1 ;; + --refmap=*|--no-refmap) + refmap=$1 ;; + -h|--help-all) + usage + ;; + --) + shift + break + ;; + *) + usage + ;; + esac + shift +done + +case "$rebase" in +preserve) + rebase=true + rebase_args=--preserve-merges + ;; +true|false|'') + ;; +*) + echo "Invalid value for --rebase, should be true, false, or preserve" + usage + exit 1 + ;; +esac + +error_on_no_merge_candidates () { + exec >&2 + + if test true = "$rebase" + then + op_type=rebase + op_prep=against + else + op_type=merge + op_prep=with + fi + + upstream=$(git config "branch.$curr_branch_short.merge") + remote=$(git config "branch.$curr_branch_short.remote") + + if [ $# -gt 1 ]; then + if [ "$rebase" = true ]; then + printf "There is no candidate for rebasing against " + else + printf "There are no candidates for merging " + fi + echo "among the refs that you just fetched." + echo "Generally this means that you provided a wildcard refspec which had no" + echo "matches on the remote end." + elif [ $# -gt 0 ] && [ "$1" != "$remote" ]; then + echo "You asked to pull from the remote '$1', but did not specify" + echo "a branch. Because this is not the default configured remote" + echo "for your current branch, you must specify a branch on the command line." + elif [ -z "$curr_branch" -o -z "$upstream" ]; then + . git-parse-remote + error_on_missing_default_upstream "pull" $op_type $op_prep \ + "git pull <remote> <branch>" + else + echo "Your configuration specifies to $op_type $op_prep the ref '${upstream#refs/heads/}'" + echo "from the remote, but no such ref was fetched." + fi + exit 1 +} + +test true = "$rebase" && { + if ! git rev-parse -q --verify HEAD >/dev/null + then + # On an unborn branch + if test -f "$(git rev-parse --git-path index)" + then + die "$(gettext "updating an unborn branch with changes added to the index")" + fi + else + require_clean_work_tree "pull with rebase" "Please commit or stash them." + fi + oldremoteref= && + test -n "$curr_branch" && + . git-parse-remote && + remoteref="$(get_remote_merge_branch "$@" 2>/dev/null)" && + oldremoteref=$(git merge-base --fork-point "$remoteref" $curr_branch 2>/dev/null) +} +orig_head=$(git rev-parse -q --verify HEAD) +git fetch $verbosity $progress $dry_run $recurse_submodules $all $append \ +${upload_pack:+"$upload_pack"} $force $tags $prune $keep $depth $unshallow $update_shallow \ +$refmap --update-head-ok "$@" || exit 1 +test -z "$dry_run" || exit 0 + +curr_head=$(git rev-parse -q --verify HEAD) +if test -n "$orig_head" && test "$curr_head" != "$orig_head" +then + # The fetch involved updating the current branch. + + # The working tree and the index file is still based on the + # $orig_head commit, but we are merging into $curr_head. + # First update the working tree to match $curr_head. + + eval_gettextln "Warning: fetch updated the current branch head. +Warning: fast-forwarding your working tree from +Warning: commit \$orig_head." >&2 + git update-index -q --refresh + git read-tree -u -m "$orig_head" "$curr_head" || + die "$(eval_gettext "Cannot fast-forward your working tree. +After making sure that you saved anything precious from +$ git diff \$orig_head +output, run +$ git reset --hard +to recover.")" + +fi + +merge_head=$(sed -e '/ not-for-merge /d' \ + -e 's/ .*//' "$GIT_DIR"/FETCH_HEAD | \ + tr '\012' ' ') + +case "$merge_head" in +'') + error_on_no_merge_candidates "$@" + ;; +?*' '?*) + if test -z "$orig_head" + then + die "$(gettext "Cannot merge multiple branches into empty head")" + fi + if test true = "$rebase" + then + die "$(gettext "Cannot rebase onto multiple branches")" + fi + ;; +esac + +# Pulling into unborn branch: a shorthand for branching off +# FETCH_HEAD, for lazy typers. +if test -z "$orig_head" +then + # Two-way merge: we claim the index is based on an empty tree, + # and try to fast-forward to HEAD. This ensures we will not + # lose index/worktree changes that the user already made on + # the unborn branch. + empty_tree=4b825dc642cb6eb9a060e54bf8d69288fbee4904 + git read-tree -m -u $empty_tree $merge_head && + git update-ref -m "initial pull" HEAD $merge_head "$curr_head" + exit +fi + +if test true = "$rebase" +then + o=$(git show-branch --merge-base $curr_branch $merge_head $oldremoteref) + if test "$oldremoteref" = "$o" + then + unset oldremoteref + fi +fi + +case "$rebase" in +true) + eval="git-rebase $diffstat $strategy_args $merge_args $rebase_args $verbosity" + eval="$eval $gpg_sign_args" + eval="$eval --onto $merge_head ${oldremoteref:-$merge_head}" + ;; +*) + eval="git-merge $diffstat $no_commit $verify_signatures $edit $squash $no_ff $ff_only" + eval="$eval $log_arg $strategy_args $merge_args $verbosity $progress" + eval="$eval $gpg_sign_args" + eval="$eval FETCH_HEAD" + ;; +esac +eval "exec $eval" diff --git a/contrib/examples/git-repack.sh b/contrib/examples/git-repack.sh new file mode 100755 index 0000000000..96e3fed326 --- /dev/null +++ b/contrib/examples/git-repack.sh @@ -0,0 +1,194 @@ +#!/bin/sh +# +# Copyright (c) 2005 Linus Torvalds +# + +OPTIONS_KEEPDASHDASH= +OPTIONS_SPEC="\ +git repack [options] +-- +a pack everything in a single pack +A same as -a, and turn unreachable objects loose +d remove redundant packs, and run git-prune-packed +f pass --no-reuse-delta to git-pack-objects +F pass --no-reuse-object to git-pack-objects +n do not run git-update-server-info +q,quiet be quiet +l pass --local to git-pack-objects +unpack-unreachable= with -A, do not loosen objects older than this + Packing constraints +window= size of the window used for delta compression +window-memory= same as the above, but limit memory size instead of entries count +depth= limits the maximum delta depth +max-pack-size= maximum size of each packfile +" +SUBDIRECTORY_OK='Yes' +. git-sh-setup + +no_update_info= all_into_one= remove_redundant= unpack_unreachable= +local= no_reuse= extra= +while test $# != 0 +do + case "$1" in + -n) no_update_info=t ;; + -a) all_into_one=t ;; + -A) all_into_one=t + unpack_unreachable=--unpack-unreachable ;; + --unpack-unreachable) + unpack_unreachable="--unpack-unreachable=$2"; shift ;; + -d) remove_redundant=t ;; + -q) GIT_QUIET=t ;; + -f) no_reuse=--no-reuse-delta ;; + -F) no_reuse=--no-reuse-object ;; + -l) local=--local ;; + --max-pack-size|--window|--window-memory|--depth) + extra="$extra $1=$2"; shift ;; + --) shift; break;; + *) usage ;; + esac + shift +done + +case "$(git config --bool repack.usedeltabaseoffset || echo true)" in +true) + extra="$extra --delta-base-offset" ;; +esac + +PACKDIR="$GIT_OBJECT_DIRECTORY/pack" +PACKTMP="$PACKDIR/.tmp-$$-pack" +rm -f "$PACKTMP"-* +trap 'rm -f "$PACKTMP"-*' 0 1 2 3 15 + +# There will be more repacking strategies to come... +case ",$all_into_one," in +,,) + args='--unpacked --incremental' + ;; +,t,) + args= existing= + if [ -d "$PACKDIR" ]; then + for e in `cd "$PACKDIR" && find . -type f -name '*.pack' \ + | sed -e 's/^\.\///' -e 's/\.pack$//'` + do + if [ -e "$PACKDIR/$e.keep" ]; then + : keep + else + existing="$existing $e" + fi + done + if test -n "$existing" && test -n "$unpack_unreachable" && \ + test -n "$remove_redundant" + then + # This may have arbitrary user arguments, so we + # have to protect it against whitespace splitting + # when it gets run as "pack-objects $args" later. + # Fortunately, we know it's an approxidate, so we + # can just use dots instead. + args="$args $(echo "$unpack_unreachable" | tr ' ' .)" + fi + fi + ;; +esac + +mkdir -p "$PACKDIR" || exit + +args="$args $local ${GIT_QUIET:+-q} $no_reuse$extra" +names=$(git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog $args </dev/null "$PACKTMP") || + exit 1 +if [ -z "$names" ]; then + say Nothing new to pack. +fi + +# Ok we have prepared all new packfiles. + +# First see if there are packs of the same name and if so +# if we can move them out of the way (this can happen if we +# repacked immediately after packing fully. +rollback= +failed= +for name in $names +do + for sfx in pack idx + do + file=pack-$name.$sfx + test -f "$PACKDIR/$file" || continue + rm -f "$PACKDIR/old-$file" && + mv "$PACKDIR/$file" "$PACKDIR/old-$file" || { + failed=t + break + } + rollback="$rollback $file" + done + test -z "$failed" || break +done + +# If renaming failed for any of them, roll the ones we have +# already renamed back to their original names. +if test -n "$failed" +then + rollback_failure= + for file in $rollback + do + mv "$PACKDIR/old-$file" "$PACKDIR/$file" || + rollback_failure="$rollback_failure $file" + done + if test -n "$rollback_failure" + then + echo >&2 "WARNING: Some packs in use have been renamed by" + echo >&2 "WARNING: prefixing old- to their name, in order to" + echo >&2 "WARNING: replace them with the new version of the" + echo >&2 "WARNING: file. But the operation failed, and" + echo >&2 "WARNING: attempt to rename them back to their" + echo >&2 "WARNING: original names also failed." + echo >&2 "WARNING: Please rename them in $PACKDIR manually:" + for file in $rollback_failure + do + echo >&2 "WARNING: old-$file -> $file" + done + fi + exit 1 +fi + +# Now the ones with the same name are out of the way... +fullbases= +for name in $names +do + fullbases="$fullbases pack-$name" + chmod a-w "$PACKTMP-$name.pack" + chmod a-w "$PACKTMP-$name.idx" + mv -f "$PACKTMP-$name.pack" "$PACKDIR/pack-$name.pack" && + mv -f "$PACKTMP-$name.idx" "$PACKDIR/pack-$name.idx" || + exit +done + +# Remove the "old-" files +for name in $names +do + rm -f "$PACKDIR/old-pack-$name.idx" + rm -f "$PACKDIR/old-pack-$name.pack" +done + +# End of pack replacement. + +if test "$remove_redundant" = t +then + # We know $existing are all redundant. + if [ -n "$existing" ] + then + ( cd "$PACKDIR" && + for e in $existing + do + case " $fullbases " in + *" $e "*) ;; + *) rm -f "$e.pack" "$e.idx" "$e.keep" ;; + esac + done + ) + fi + git prune-packed ${GIT_QUIET:+-q} +fi + +case "$no_update_info" in +t) : ;; +*) git update-server-info ;; +esac diff --git a/contrib/examples/git-reset.sh b/contrib/examples/git-reset.sh index bafeb52cd1..cb1bbf3b90 100755 --- a/contrib/examples/git-reset.sh +++ b/contrib/examples/git-reset.sh @@ -40,7 +40,7 @@ case "$1" in --) shift ;; esac # git reset --mixed tree [--] paths... can be used to # load chosen paths from the tree into the index without -# affecting the working tree nor HEAD. +# affecting the working tree or HEAD. if test $# != 0 then test "$reset_type" = "--mixed" || @@ -60,7 +60,7 @@ then update=-u fi -# Soft reset does not touch the index file nor the working tree +# Soft reset does not touch the index file or the working tree # at all, but requires them in a good order. Other resets reset # the index file to the tree object we are switching to. if test "$reset_type" = "--soft" diff --git a/contrib/examples/git-resolve.sh b/contrib/examples/git-resolve.sh index 8f98142f77..70fdc27b72 100755 --- a/contrib/examples/git-resolve.sh +++ b/contrib/examples/git-resolve.sh @@ -75,8 +75,8 @@ case "$common" in GIT_INDEX_FILE=$G git read-tree -m $c $head $merge \ 2>/dev/null || continue # Count the paths that are unmerged. - cnt=`GIT_INDEX_FILE=$G git ls-files --unmerged | wc -l` - if test $best_cnt -le 0 -o $cnt -le $best_cnt + cnt=$(GIT_INDEX_FILE=$G git ls-files --unmerged | wc -l) + if test $best_cnt -le 0 || test $cnt -le $best_cnt then best=$c best_cnt=$cnt diff --git a/contrib/examples/git-revert.sh b/contrib/examples/git-revert.sh index 6bf155cbdb..7e2aad5491 100755 --- a/contrib/examples/git-revert.sh +++ b/contrib/examples/git-revert.sh @@ -137,7 +137,7 @@ cherry-pick) q }' - logmsg=`git show -s --pretty=raw --encoding="$encoding" "$commit"` + logmsg=$(git show -s --pretty=raw --encoding="$encoding" "$commit") set_author_env=`echo "$logmsg" | LANG=C LC_ALL=C sed -ne "$pick_author_script"` eval "$set_author_env" diff --git a/contrib/examples/git-svnimport.txt b/contrib/examples/git-svnimport.txt index 3bb871e42f..3f0a9c33b5 100644 --- a/contrib/examples/git-svnimport.txt +++ b/contrib/examples/git-svnimport.txt @@ -176,4 +176,4 @@ Documentation by Matthias Urlichs <smurf@smurf.noris.de>. GIT --- -Part of the gitlink:git[7] suite +Part of the linkgit:git[7] suite diff --git a/contrib/examples/git-tag.sh b/contrib/examples/git-tag.sh index 2c15bc955b..1bd8f3c58d 100755 --- a/contrib/examples/git-tag.sh +++ b/contrib/examples/git-tag.sh @@ -156,7 +156,7 @@ prev=0000000000000000000000000000000000000000 if git show-ref --verify --quiet -- "refs/tags/$name" then test -n "$force" || die "tag '$name' already exists" - prev=`git rev-parse "refs/tags/$name"` + prev=$(git rev-parse "refs/tags/$name") fi shift git check-ref-format "tags/$name" || diff --git a/contrib/examples/git-whatchanged.sh b/contrib/examples/git-whatchanged.sh new file mode 100755 index 0000000000..2edbdc6d99 --- /dev/null +++ b/contrib/examples/git-whatchanged.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +USAGE='[-p] [--max-count=<n>] [<since>..<limit>] [--pretty=<format>] [-m] [git-diff-tree options] [git-rev-list options]' +SUBDIRECTORY_OK='Yes' +. git-sh-setup + +diff_tree_flags=$(git-rev-parse --sq --no-revs --flags "$@") || exit +case "$0" in +*whatchanged) + count= + test -z "$diff_tree_flags" && + diff_tree_flags=$(git config --get whatchanged.difftree) + diff_tree_default_flags='-c -M --abbrev' ;; +*show) + count=-n1 + test -z "$diff_tree_flags" && + diff_tree_flags=$(git config --get show.difftree) + diff_tree_default_flags='--cc --always' ;; +esac +test -z "$diff_tree_flags" && + diff_tree_flags="$diff_tree_default_flags" + +rev_list_args=$(git-rev-parse --sq --default HEAD --revs-only "$@") && +diff_tree_args=$(git-rev-parse --sq --no-revs --no-flags "$@") && + +eval "git-rev-list $count $rev_list_args" | +eval "git-diff-tree --stdin --pretty -r $diff_tree_flags $diff_tree_args" | +LESS="$LESS -S" ${PAGER:-less} diff --git a/contrib/fast-import/import-directories.perl b/contrib/fast-import/import-directories.perl index 7f3afa5ac4..4dec1f18e4 100755 --- a/contrib/fast-import/import-directories.perl +++ b/contrib/fast-import/import-directories.perl @@ -109,8 +109,8 @@ was available previously is not included in this revision, it will be removed. If an on-disk revision is incomplete, you can point to files from -a previous revision. There are no restriction as to where the source -files are located, nor to the names of them. +a previous revision. There are no restrictions on where the source +files are located, nor on their names. [3.files] ; the key is the path inside the repository, the value is the path diff --git a/contrib/git-resurrect.sh b/contrib/git-resurrect.sh index a4ed4c3c62..d7e97bbc76 100755 --- a/contrib/git-resurrect.sh +++ b/contrib/git-resurrect.sh @@ -10,6 +10,7 @@ is rather slow but allows you to resurrect other people's topic branches." OPTIONS_KEEPDASHDASH= +OPTIONS_STUCKLONG= OPTIONS_SPEC="\ git resurrect $USAGE -- diff --git a/contrib/gitview/gitview b/contrib/gitview/gitview index 4c99dfb903..4e23c650fe 100755 --- a/contrib/gitview/gitview +++ b/contrib/gitview/gitview @@ -1205,7 +1205,7 @@ class GitView(object): #The first parent always continue on the same line try: - # check we alreay have the value + # check we already have the value tmp_node_pos = self.nodepos[commit.parent_sha1[0]] except KeyError: self.colours[commit.parent_sha1[0]] = colour diff --git a/contrib/gitview/gitview.txt b/contrib/gitview/gitview.txt index 9e12f97842..7b5f9002b9 100644 --- a/contrib/gitview/gitview.txt +++ b/contrib/gitview/gitview.txt @@ -28,7 +28,7 @@ OPTIONS <args>:: - All the valid option for gitlink:git-rev-list[1]. + All the valid option for linkgit:git-rev-list[1]. Key Bindings ------------ diff --git a/contrib/hg-to-git/hg-to-git.py b/contrib/hg-to-git/hg-to-git.py index 232625a7b7..60dec86d37 100755 --- a/contrib/hg-to-git/hg-to-git.py +++ b/contrib/hg-to-git/hg-to-git.py @@ -225,7 +225,7 @@ for cset in range(int(tip) + 1): os.system('git ls-files -x .hg --deleted | git update-index --remove --stdin') # commit - os.system(getgitenv(user, date) + 'git commit --allow-empty -a -F %s' % filecomment) + os.system(getgitenv(user, date) + 'git commit --allow-empty --allow-empty-message -a -F %s' % filecomment) os.unlink(filecomment) # tag diff --git a/contrib/hooks/multimail/CHANGES b/contrib/hooks/multimail/CHANGES new file mode 100644 index 0000000000..6bb12306b8 --- /dev/null +++ b/contrib/hooks/multimail/CHANGES @@ -0,0 +1,86 @@ +Release 1.1.1 (bugfix-only release) +=================================== + +* The SMTP mailer was not working with Python 2.4. + +Release 1.1.0 +============= + +* When a single commit is pushed, omit the reference changed email. + Set multimailhook.combineWhenSingleCommit to false to disable this + new feature. + +* In gitolite environments, the pusher's email address can be used as + the From address by creating a specially formatted comment block in + gitolite.conf (see multimailhook.from in README). + +* Support for SMTP authentication and SSL/TLS encryption was added, + see smtpUser, smtpPass, smtpEncryption in README. + +* A new option scanCommitForCc was added to allow git-multimail to + search the commit message for 'Cc: ...' lines, and add the + corresponding emails in Cc. + +* If $USER is not set, use the variable $USERNAME. This is needed on + Windows platform to recognize the pusher. + +* The emailPrefix variable can now be set to an empty string to remove + the prefix. + +* A short tutorial was added in doc/gitolite.rst to set up + git-multimail with gitolite. + +* The post-receive file was renamed to post-receive.example. It has + always been an example (the standard way to call git-multimail is to + call git_multimail.py), but it was unclear to many users. + +* A new refchangeShowGraph option was added to make it possible to + include both a graph and a log in the summary emails. The options + to control the graph formatting can be set via the new graphOpts + option. + +* New option --force-send was added to disable new commit detection + for update hook. One use-case is to run git_multimail.py after + running "git fetch" to send emails about commits that have just been + fetched (the detection of new commits was unreliable in this mode). + +* The testing infrastructure was considerably improved (continuous + integration with travis-ci, automatic check of PEP8 and RST syntax, + many improvements to the test scripts). + +This version has been tested with Python 2.4 to 2.7, and Git 1.7.1 to +2.4. + +Release 1.0.0 +============= + +* Fix encoding of non-ASCII email addresses in email headers. + +* Fix backwards-compatibility bugs for older Python 2.x versions. + +* Fix a backwards-compatibility bug for Git 1.7.1. + +* Add an option commitDiffOpts to customize logs for revisions. + +* Pass "-oi" to sendmail by default to prevent premature termination + on a line containing only ".". + +* Stagger email "Date:" values in an attempt to help mail clients + thread the emails in the right order. + +* If a mailing list setting is missing, just skip sending the + corresponding email (with a warning) instead of failing. + +* Add a X-Git-Host header that can be used for email filtering. + +* Allow the sender's fully-qualified domain name to be configured. + +* Minor documentation improvements. + +* Add this CHANGES file. + + +Release 0.9.0 +============= + +* Initial release. diff --git a/contrib/hooks/multimail/README b/contrib/hooks/multimail/README new file mode 100644 index 0000000000..e552c90c45 --- /dev/null +++ b/contrib/hooks/multimail/README @@ -0,0 +1,594 @@ +git-multimail Version 1.1.1 +=========================== + +.. image:: https://travis-ci.org/git-multimail/git-multimail.svg?branch=master + :target: https://travis-ci.org/git-multimail/git-multimail + +git-multimail is a tool for sending notification emails on pushes to a +Git repository. It includes a Python module called git_multimail.py, +which can either be used as a hook script directly or can be imported +as a Python module into another script. + +git-multimail is derived from the Git project's old +contrib/hooks/post-receive-email, and is mostly compatible with that +script. See README.migrate-from-post-receive-email for details about +the differences and for how to migrate from post-receive-email to +git-multimail. + +git-multimail, like the rest of the Git project, is licensed under +GPLv2 (see the COPYING file for details). + +Please note: although, as a convenience, git-multimail may be +distributed along with the main Git project, development of +git-multimail takes place in its own, separate project. See section +"Getting involved" below for more information. + + +By default, for each push received by the repository, git-multimail: + +1. Outputs one email summarizing each reference that was changed. + These "reference change" (called "refchange" below) emails describe + the nature of the change (e.g., was the reference created, deleted, + fast-forwarded, etc.) and include a one-line summary of each commit + that was added to the reference. + +2. Outputs one email for each new commit that was introduced by the + reference change. These "commit" emails include a list of the + files changed by the commit, followed by the diffs of files + modified by the commit. The commit emails are threaded to the + corresponding reference change email via "In-Reply-To". This style + (similar to the "git format-patch" style used on the Git mailing + list) makes it easy to scan through the emails, jump to patches + that need further attention, and write comments about specific + commits. Commits are handled in reverse topological order (i.e., + parents shown before children). For example:: + + [git] branch master updated + + [git] 01/08: doc: fix xref link from api docs to manual pages + + [git] 02/08: api-credentials.txt: show the big picture first + + [git] 03/08: api-credentials.txt: mention credential.helper explicitly + + [git] 04/08: api-credentials.txt: add "see also" section + + [git] 05/08: t3510 (cherry-pick-sequence): add missing '&&' + + [git] 06/08: Merge branch 'rr/maint-t3510-cascade-fix' + + [git] 07/08: Merge branch 'mm/api-credentials-doc' + + [git] 08/08: Git 1.7.11-rc2 + + Each commit appears in exactly one commit email, the first time + that it is pushed to the repository. If a commit is later merged + into another branch, then a one-line summary of the commit is + included in the reference change email (as usual), but no + additional commit email is generated. + + By default, reference change emails have their "Reply-To" field set + to the person who pushed the change, and commit emails have their + "Reply-To" field set to the author of the commit. + +3. Output one "announce" mail for each new annotated tag, including + information about the tag and optionally a shortlog describing the + changes since the previous tag. Such emails might be useful if you + use annotated tags to mark releases of your project. + + +Requirements +------------ + +* Python 2.x, version 2.4 or later. No non-standard Python modules + are required. git-multimail does *not* currently work with Python + 3.x. + + The example scripts invoke Python using the following shebang line + (following PEP 394 [1]_):: + + #! /usr/bin/env python2 + + If your system's Python2 interpreter is not in your PATH or is not + called ``python2``, you can change the lines accordingly. Or you can + invoke the Python interpreter explicitly, for example via a tiny + shell script like:: + + #! /bin/sh + /usr/local/bin/python /path/to/git_multimail.py "$@" + +* The ``git`` command must be in your PATH. git-multimail is known to + work with Git versions back to 1.7.1. (Earlier versions have not + been tested; if you do so, please report your results.) + +* To send emails using the default configuration, a standard sendmail + program must be located at '/usr/sbin/sendmail' or + '/usr/lib/sendmail' and must be configured correctly to send emails. + If this is not the case, set multimailhook.sendmailCommand, or see + the multimailhook.mailer configuration variable below for how to + configure git-multimail to send emails via an SMTP server. + + +Invocation +---------- + +git_multimail.py is designed to be used as a ``post-receive`` hook in a +Git repository (see githooks(5)). Link or copy it to +$GIT_DIR/hooks/post-receive within the repository for which email +notifications are desired. Usually it should be installed on the +central repository for a project, to which all commits are eventually +pushed. + +For use on pre-v1.5.1 Git servers, git_multimail.py can also work as +an ``update`` hook, taking its arguments on the command line. To use +this script in this manner, link or copy it to $GIT_DIR/hooks/update. +Please note that the script is not completely reliable in this mode +[2]_. + +Alternatively, git_multimail.py can be imported as a Python module +into your own Python post-receive script. This method is a bit more +work, but allows the behavior of the hook to be customized using +arbitrary Python code. For example, you can use a custom environment +(perhaps inheriting from GenericEnvironment or GitoliteEnvironment) to + +* change how the user who did the push is determined + +* read users' email addresses from an LDAP server or from a database + +* decide which users should be notified about which commits based on + the contents of the commits (e.g., for users who want to be notified + only about changes affecting particular files or subdirectories) + +Or you can change how emails are sent by writing your own Mailer +class. The ``post-receive`` script in this directory demonstrates how +to use git_multimail.py as a Python module. (If you make interesting +changes of this type, please consider sharing them with the +community.) + + +Configuration +------------- + +By default, git-multimail mostly takes its configuration from the +following ``git config`` settings: + +multimailhook.environment + + This describes the general environment of the repository. + Currently supported values: + + * generic + + the username of the pusher is read from $USER or $USERNAME and + the repository name is derived from the repository's path. + + * gitolite + + the username of the pusher is read from $GL_USER, the repository + name is read from $GL_REPO, and the From: header value is + optionally read from gitolite.conf (see multimailhook.from). + + For more information about gitolite and git-multimail, read + doc/gitolite.rst + + If neither of these environments is suitable for your setup, then + you can implement a Python class that inherits from Environment + and instantiate it via a script that looks like the example + post-receive script. + + The environment value can be specified on the command line using + the --environment option. If it is not specified on the command + line or by multimailhook.environment, then it defaults to + ``gitolite`` if the environment contains variables $GL_USER and + $GL_REPO; otherwise ``generic``. + +multimailhook.repoName + + A short name of this Git repository, to be used in various places + in the notification email text. The default is to use $GL_REPO + for gitolite repositories, or otherwise to derive this value from + the repository path name. + +multimailhook.mailingList + + The list of email addresses to which notification emails should be + sent, as RFC 2822 email addresses separated by commas. This + configuration option can be multivalued. Leave it unset or set it + to the empty string to not send emails by default. The next few + settings can be used to configure specific address lists for + specific types of notification email. + +multimailhook.refchangeList + + The list of email addresses to which summary emails about + reference changes should be sent, as RFC 2822 email addresses + separated by commas. This configuration option can be + multivalued. The default is the value in + multimailhook.mailingList. Set this value to the empty string to + prevent reference change emails from being sent even if + multimailhook.mailingList is set. + +multimailhook.announceList + + The list of email addresses to which emails about new annotated + tags should be sent, as RFC 2822 email addresses separated by + commas. This configuration option can be multivalued. The + default is the value in multimailhook.refchangeList or + multimailhook.mailingList. Set this value to the empty string to + prevent annotated tag announcement emails from being sent even if + one of the other values is set. + +multimailhook.commitList + + The list of email addresses to which emails about individual new + commits should be sent, as RFC 2822 email addresses separated by + commas. This configuration option can be multivalued. The + default is the value in multimailhook.mailingList. Set this value + to the empty string to prevent notification emails about + individual commits from being sent even if + multimailhook.mailingList is set. + +multimailhook.announceShortlog + + If this option is set to true, then emails about changes to + annotated tags include a shortlog of changes since the previous + tag. This can be useful if the annotated tags represent releases; + then the shortlog will be a kind of rough summary of what has + happened since the last release. But if your tagging policy is + not so straightforward, then the shortlog might be confusing + rather than useful. Default is false. + +multimailhook.refchangeShowGraph + + If this option is set to true, then summary emails about reference + changes will additionally include: + + * a graph of the added commits (if any) + + * a graph of the discarded commits (if any) + + The log is generated by running ``git log --graph`` with the options + specified in graphOpts. The default is false. + +multimailhook.refchangeShowLog + + If this option is set to true, then summary emails about reference + changes will include a detailed log of the added commits in + addition to the one line summary. The log is generated by running + ``git log`` with the options specified in multimailhook.logOpts. + Default is false. + +multimailhook.mailer + + This option changes the way emails are sent. Accepted values are: + + - sendmail (the default): use the command ``/usr/sbin/sendmail`` or + ``/usr/lib/sendmail`` (or sendmailCommand, if configured). This + mode can be further customized via the following options: + + * multimailhook.sendmailCommand + + The command used by mailer ``sendmail`` to send emails. Shell + quoting is allowed in the value of this setting, but remember that + Git requires double-quotes to be escaped; e.g.:: + + git config multimailhook.sendmailcommand '/usr/sbin/sendmail -oi -t -F \"Git Repo\"' + + Default is '/usr/sbin/sendmail -oi -t' or + '/usr/lib/sendmail -oi -t' (depending on which file is + present and executable). + + * multimailhook.envelopeSender + + If set then pass this value to sendmail via the -f option to set + the envelope sender address. + + - smtp: use Python's smtplib. This is useful when the sendmail + command is not available on the system. This mode can be + further customized via the following options: + + * multimailhook.smtpServer + + The name of the SMTP server to connect to. The value can + also include a colon and a port number; e.g., + ``mail.example.com:25``. Default is 'localhost' using port 25. + + * multimailhook.smtpUser + * multimailhook.smtpPass + + Server username and password. Required if smtpEncryption is 'ssl'. + Note that the username and password currently need to be + set cleartext in the configuration file, which is not + recommended. If you need to use this option, be sure your + configuration file is read-only. + + * multimailhook.envelopeSender + + The sender address to be passed to the SMTP server. If + unset, then the value of multimailhook.from is used. + + * multimailhook.smtpServerTimeout + + Timeout in seconds. + + * multimailhook.smtpEncryption + + Set the security type. Allowed values: none, ssl. + Default=none. + + * multimailhook.smtpServerDebugLevel + + Integer number. Set to greater than 0 to activate debugging. + +multimailhook.from + + If set, use this value in the From: field of generated emails. If + unset, the value of the From: header is determined as follows: + + 1. (gitolite environment only) Parse gitolite.conf, looking for a + block of comments that looks like this:: + + # BEGIN USER EMAILS + # username Firstname Lastname <email@example.com> + # END USER EMAILS + + If that block exists, and there is a line between the BEGIN + USER EMAILS and END USER EMAILS lines where the first field + matches the gitolite username ($GL_USER), use the rest of the + line for the From: header. + + 2. If the user.email configuration setting is set, use its value + (and the value of user.name, if set). + + 3. Use the value of multimailhook.envelopeSender. + +multimailhook.administrator + + The name and/or email address of the administrator of the Git + repository; used in FOOTER_TEMPLATE. Default is + multimailhook.envelopesender if it is set; otherwise a generic + string is used. + +multimailhook.emailPrefix + + All emails have this string prepended to their subjects, to aid + email filtering (though filtering based on the X-Git-* email + headers is probably more robust). Default is the short name of + the repository in square brackets; e.g., ``[myrepo]``. Set this + value to the empty string to suppress the email prefix. + +multimailhook.emailMaxLines + + The maximum number of lines that should be included in the body of + a generated email. If not specified, there is no limit. Lines + beyond the limit are suppressed and counted, and a final line is + added indicating the number of suppressed lines. + +multimailhook.emailMaxLineLength + + The maximum length of a line in the email body. Lines longer than + this limit are truncated to this length with a trailing `` [...]`` + added to indicate the missing text. The default is 500, because + (a) diffs with longer lines are probably from binary files, for + which a diff is useless, and (b) even if a text file has such long + lines, the diffs are probably unreadable anyway. To disable line + truncation, set this option to 0. + +multimailhook.maxCommitEmails + + The maximum number of commit emails to send for a given change. + When the number of patches is larger that this value, only the + summary refchange email is sent. This can avoid accidental + mailbombing, for example on an initial push. To disable commit + emails limit, set this option to 0. The default is 500. + +multimailhook.emailStrictUTF8 + + If this boolean option is set to `true`, then the main part of the + email body is forced to be valid UTF-8. Any characters that are + not valid UTF-8 are converted to the Unicode replacement + character, U+FFFD. The default is `true`. + +multimailhook.diffOpts + + Options passed to ``git diff-tree`` when generating the summary + information for ReferenceChange emails. Default is ``--stat + --summary --find-copies-harder``. Add -p to those options to + include a unified diff of changes in addition to the usual summary + output. Shell quoting is allowed; see multimailhook.logOpts for + details. + +multimailhook.graphOpts + + Options passed to ``git log --graph`` when generating graphs for the + reference change summary emails (used only if refchangeShowGraph + is true). The default is '--oneline --decorate'. + + Shell quoting is allowed; see logOpts for details. + +multimailhook.logOpts + + Options passed to ``git log`` to generate additional info for + reference change emails (used only if refchangeShowLog is set). + For example, adding -p will show each commit's complete diff. The + default is empty. + + Shell quoting is allowed; for example, a log format that contains + spaces can be specified using something like:: + + git config multimailhook.logopts '--pretty=format:"%h %aN <%aE>%n%s%n%n%b%n"' + + If you want to set this by editing your configuration file + directly, remember that Git requires double-quotes to be escaped + (see git-config(1) for more information):: + + [multimailhook] + logopts = --pretty=format:\"%h %aN <%aE>%n%s%n%n%b%n\" + +multimailhook.commitLogOpts + + Options passed to ``git log`` to generate additional info for + revision change emails. For example, adding --ignore-all-spaces + will suppress whitespace changes. The default options are ``-C + --stat -p --cc``. Shell quoting is allowed; see + multimailhook.logOpts for details. + +multimailhook.emailDomain + + Domain name appended to the username of the person doing the push + to convert it into an email address + (via ``"%s@%s" % (username, emaildomain)``). More complicated + schemes can be implemented by overriding Environment and + overriding its get_pusher_email() method. + +multimailhook.replyTo +multimailhook.replyToCommit +multimailhook.replyToRefchange + + Addresses to use in the Reply-To: field for commit emails + (replyToCommit) and refchange emails (replyToRefchange). + multimailhook.replyTo is used as default when replyToCommit or + replyToRefchange is not set. The value for these variables can be + either: + + - An email address, which will be used directly. + + - The value `pusher`, in which case the pusher's address (if + available) will be used. This is the default for refchange + emails. + + - The value `author` (meaningful only for replyToCommit), in which + case the commit author's address will be used. This is the + default for commit emails. + + - The value `none`, in which case the Reply-To: field will be + omitted. + +multimailhook.quiet + + Do not output the list of email recipients from the hook + +multimailhook.stdout + + For debugging, send emails to stdout rather than to the + mailer. Equivalent to the --stdout command line option + +multimailhook.scanCommitForCc + + If this option is set to true, than recipients from lines in commit body + that starts with ``CC:`` will be added to CC list. + Default: false + +multimailhook.combineWhenSingleCommit + + If this option is set to true and a single new commit is pushed to + a branch, combine the summary and commit email messages into a + single email. + Default: true + + +Email filtering aids +-------------------- + +All emails include extra headers to enable fine tuned filtering and +give information for debugging. All emails include the headers +``X-Git-Host``, ``X-Git-Repo``, ``X-Git-Refname``, and ``X-Git-Reftype``. +ReferenceChange emails also include headers ``X-Git-Oldrev`` and ``X-Git-Newrev``; +Revision emails also include header ``X-Git-Rev``. + + +Customizing email contents +-------------------------- + +git-multimail mostly generates emails by expanding templates. The +templates can be customized. To avoid the need to edit +git_multimail.py directly, the preferred way to change the templates +is to write a separate Python script that imports git_multimail.py as +a module, then replaces the templates in place. See the provided +post-receive script for an example of how this is done. + + +Customizing git-multimail for your environment +---------------------------------------------- + +git-multimail is mostly customized via an "environment" that describes +the local environment in which Git is running. Two types of +environment are built in: + +* GenericEnvironment: a stand-alone Git repository. + +* GitoliteEnvironment: a Git repository that is managed by gitolite + [3]_. For such repositories, the identity of the pusher is read from + environment variable $GL_USER, the name of the repository is read + from $GL_REPO (if it is not overridden by multimailhook.reponame), + and the From: header value is optionally read from gitolite.conf + (see multimailhook.from). + +By default, git-multimail assumes GitoliteEnvironment if $GL_USER and +$GL_REPO are set, and otherwise assumes GenericEnvironment. +Alternatively, you can choose one of these two environments explicitly +by setting a ``multimailhook.environment`` config setting (which can +have the value `generic` or `gitolite`) or by passing an --environment +option to the script. + +If you need to customize the script in ways that are not supported by +the existing environments, you can define your own environment class +class using arbitrary Python code. To do so, you need to import +git_multimail.py as a Python module, as demonstrated by the example +post-receive script. Then implement your environment class; it should +usually inherit from one of the existing Environment classes and +possibly one or more of the EnvironmentMixin classes. Then set the +``environment`` variable to an instance of your own environment class +and pass it to ``run_as_post_receive_hook()``. + +The standard environment classes, GenericEnvironment and +GitoliteEnvironment, are in fact themselves put together out of a +number of mixin classes, each of which handles one aspect of the +customization. For the finest control over your configuration, you +can specify exactly which mixin classes your own environment class +should inherit from, and override individual methods (or even add your +own mixin classes) to implement entirely new behaviors. If you +implement any mixins that might be useful to other people, please +consider sharing them with the community! + + +Getting involved +---------------- + +git-multimail is an open-source project, built by volunteers. We would +welcome your help! + +The current maintainers are Michael Haggerty <mhagger@alum.mit.edu> +and Matthieu Moy <matthieu.moy@grenoble-inp.fr>. + +Please note that although a copy of git-multimail is distributed in +the "contrib" section of the main Git project, development takes place +in a separate git-multimail repository on GitHub: + + https://github.com/git-multimail/git-multimail + +Whenever enough changes to git-multimail have accumulated, a new +code-drop of git-multimail will be submitted for inclusion in the Git +project. + +We use the GitHub issue tracker to keep track of bugs and feature +requests, and we use GitHub pull requests to exchange patches (though, +if you prefer, you can send patches via the Git mailing list with CC +to the maintainers). Please sign off your patches as per the Git +project practice. + +General discussion of git-multimail can take place on the main Git +mailing list, + + git@vger.kernel.org + +Please CC emails regarding git-multimail to the maintainers so that we +don't overlook them. + + +Footnotes +--------- + +.. [1] http://www.python.org/dev/peps/pep-0394/ + +.. [2] Because of the way information is passed to update hooks, the + script's method of determining whether a commit has already + been seen does not work when it is used as an ``update`` script. + In particular, no notification email will be generated for a + new commit that is added to multiple references in the same + push. A workaround is to use --force-send to force sending the + emails. + +.. [3] https://github.com/sitaramc/gitolite diff --git a/contrib/hooks/multimail/README.Git b/contrib/hooks/multimail/README.Git new file mode 100644 index 0000000000..f5d59a8d31 --- /dev/null +++ b/contrib/hooks/multimail/README.Git @@ -0,0 +1,15 @@ +This copy of git-multimail is distributed as part of the "contrib" +section of the Git project as a convenience to Git users. +git-multimail is developed as an independent project at the following +website: + + https://github.com/git-multimail/git-multimail + +The version in this directory was obtained from the upstream project +on July 03 2015 and consists of the "git-multimail" subdirectory from +revision + + 6d6c9eb62a054143322cfaecde3949189c065b46 refs/tags/1.1.1 + +Please see the README file in this directory for information about how +to report bugs or contribute to git-multimail. diff --git a/contrib/hooks/multimail/README.migrate-from-post-receive-email b/contrib/hooks/multimail/README.migrate-from-post-receive-email new file mode 100644 index 0000000000..1e6a976699 --- /dev/null +++ b/contrib/hooks/multimail/README.migrate-from-post-receive-email @@ -0,0 +1,145 @@ +git-multimail is close to, but not exactly, a plug-in replacement for +the old Git project script contrib/hooks/post-receive-email. This +document describes the differences and explains how to configure +git-multimail to get behavior closest to that of post-receive-email. + +If you are in a hurry +===================== + +A script called migrate-mailhook-config is included with +git-multimail. If you run this script within a Git repository that is +configured to use post-receive-email, it will convert the +configuration settings into the approximate equivalent settings for +git-multimail. For more information, run + + migrate-mailhook-config --help + + +Configuration differences +========================= + +* The names of the config options for git-multimail are in namespace + "multimailhook.*" instead of "hooks.*". (Editorial comment: + post-receive-email should never have used such a generic top-level + namespace.) + +* In emails about new annotated tags, post-receive-email includes a + shortlog of all changes since the previous annotated tag. To get + this behavior with git-multimail, you need to set + multimailhook.announceshortlog to true: + + git config multimailhook.announceshortlog true + +* multimailhook.commitlist -- This is a new configuration variable. + Recipients listed here will receive a separate email for each new + commit. However, if this variable is *not* set, it defaults to the + value of multimailhook.mailinglist. Therefore, if you *don't* want + the members of multimailhook.mailinglist to receive one email per + commit, then set this value to the empty string: + + git config multimailhook.commitlist '' + +* multimailhook.emailprefix -- If this value is not set, then the + subjects of generated emails are prefixed with the short name of the + repository enclosed in square brackets; e.g., "[myrepo]". + post-receive-email defaults to prefix "[SCM]" if this option is not + set. So if you were using the old default and want to retain it + (for example, to avoid having to change your email filters), set + this variable explicitly to the old value: + + git config multimailhook.emailprefix "[SCM]" + +* The "multimailhook.showrev" configuration option is not supported. + Its main use is obsoleted by the one-email-per-commit feature of + git-multimail. + + +Other differences +================= + +This section describes other differences in the behavior of +git-multimail vs. post-receive-email. For full details, please refer +to the main README file: + +* One email per commit. For each reference change, the script first + outputs one email summarizing the reference change (including + one-line summaries of the new commits), then it outputs a separate + email for each new commit that was introduced, including patches. + These one-email-per-commit emails go to the addresses listed in + multimailhook.commitlist. post-receive-email sends only one email + for each *reference* that is changed, no matter how many commits + were added to the reference. + +* Better algorithm for detecting new commits. post-receive-email + processes one reference change at a time, which causes it to fail to + describe new commits that were included in multiple branches. For + example, if a single push adds the "*" commits in the diagram below, + then post-receive-email would never include the details of the two + commits that are common to "master" and "branch" in its + notifications. + + o---o---o---*---*---* <-- master + \ + *---* <-- branch + + git-multimail analyzes all reference modifications to determine + which commits were not present before the change, therefore avoiding + that error. + +* In reference change emails, git-multimail tells which commits have + been added to the reference vs. are entirely new to the repository, + and which commits that have been omitted from the reference + vs. entirely discarded from the repository. + +* The environment in which Git is running can be configured via an + "Environment" abstraction. + +* Built-in support for Gitolite-managed repositories. + +* Instead of using full SHA1 object names in emails, git-multimail + mostly uses abbreviated SHA1s, plus one-line log message summaries + where appropriate. + +* In the schematic diagrams that explain non-fast-forward commits, + git-multimail shows the names of the branches involved. + +* The emails generated by git-multimail include the name of the Git + repository that was modified; this is convenient for recipients who + are monitoring multiple repositories. + +* git-multimail allows the email "From" addresses to be configured. + +* The recipients lists (multimailhook.mailinglist, + multimailhook.refchangelist, multimailhook.announcelist, and + multimailhook.commitlist) can be comma-separated values and/or + multivalued settings in the config file; e.g., + + [multimailhook] + mailinglist = mr.brown@example.com, mr.black@example.com + announcelist = Him <him@example.com> + announcelist = Jim <jim@example.com> + announcelist = pop@example.com + + This might make it easier to maintain short recipients lists without + requiring full-fledged mailing list software. + +* By default, git-multimail sets email "Reply-To" headers to reply to + the pusher (for reference updates) and to the author (for commit + notifications). By default, the pusher's email address is + constructed by appending "multimailhook.emaildomain" to the pusher's + username. + +* The generated emails contain a configurable footer. By default, it + lists the name of the administrator who should be contacted to + unsubscribe from notification emails. + +* New option multimailhook.emailmaxlinelength to limit the length of + lines in the main part of the email body. The default limit is 500 + characters. + +* New option multimailhook.emailstrictutf8 to ensure that the main + part of the email body is valid UTF-8. Invalid characters are + turned into the Unicode replacement character, U+FFFD. By default + this option is turned on. + +* Written in Python. Easier to add new features. diff --git a/contrib/hooks/multimail/git_multimail.py b/contrib/hooks/multimail/git_multimail.py new file mode 100755 index 0000000000..c06ce7a515 --- /dev/null +++ b/contrib/hooks/multimail/git_multimail.py @@ -0,0 +1,3088 @@ +#! /usr/bin/env python2 + +# Copyright (c) 2015 Matthieu Moy and others +# Copyright (c) 2012-2014 Michael Haggerty and others +# Derived from contrib/hooks/post-receive-email, which is +# Copyright (c) 2007 Andy Parkins +# and also includes contributions by other authors. +# +# This file is part of git-multimail. +# +# git-multimail is free software: you can redistribute it and/or +# modify it under the terms of the GNU General Public License version +# 2 as published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see +# <http://www.gnu.org/licenses/>. + +"""Generate notification emails for pushes to a git repository. + +This hook sends emails describing changes introduced by pushes to a +git repository. For each reference that was changed, it emits one +ReferenceChange email summarizing how the reference was changed, +followed by one Revision email for each new commit that was introduced +by the reference change. + +Each commit is announced in exactly one Revision email. If the same +commit is merged into another branch in the same or a later push, then +the ReferenceChange email will list the commit's SHA1 and its one-line +summary, but no new Revision email will be generated. + +This script is designed to be used as a "post-receive" hook in a git +repository (see githooks(5)). It can also be used as an "update" +script, but this usage is not completely reliable and is deprecated. + +To help with debugging, this script accepts a --stdout option, which +causes the emails to be written to standard output rather than sent +using sendmail. + +See the accompanying README file for the complete documentation. + +""" + +import sys +import os +import re +import bisect +import socket +import subprocess +import shlex +import optparse +import smtplib +import time + +try: + from email.utils import make_msgid + from email.utils import getaddresses + from email.utils import formataddr + from email.utils import formatdate + from email.header import Header +except ImportError: + # Prior to Python 2.5, the email module used different names: + from email.Utils import make_msgid + from email.Utils import getaddresses + from email.Utils import formataddr + from email.Utils import formatdate + from email.Header import Header + + +DEBUG = False + +ZEROS = '0' * 40 +LOGBEGIN = '- Log -----------------------------------------------------------------\n' +LOGEND = '-----------------------------------------------------------------------\n' + +ADDR_HEADERS = set(['from', 'to', 'cc', 'bcc', 'reply-to', 'sender']) + +# It is assumed in many places that the encoding is uniformly UTF-8, +# so changing these constants is unsupported. But define them here +# anyway, to make it easier to find (at least most of) the places +# where the encoding is important. +(ENCODING, CHARSET) = ('UTF-8', 'utf-8') + + +REF_CREATED_SUBJECT_TEMPLATE = ( + '%(emailprefix)s%(refname_type)s %(short_refname)s created' + ' (now %(newrev_short)s)' + ) +REF_UPDATED_SUBJECT_TEMPLATE = ( + '%(emailprefix)s%(refname_type)s %(short_refname)s updated' + ' (%(oldrev_short)s -> %(newrev_short)s)' + ) +REF_DELETED_SUBJECT_TEMPLATE = ( + '%(emailprefix)s%(refname_type)s %(short_refname)s deleted' + ' (was %(oldrev_short)s)' + ) + +COMBINED_REFCHANGE_REVISION_SUBJECT_TEMPLATE = ( + '%(emailprefix)s%(refname_type)s %(short_refname)s updated: %(oneline)s' + ) + +REFCHANGE_HEADER_TEMPLATE = """\ +Date: %(send_date)s +To: %(recipients)s +Subject: %(subject)s +MIME-Version: 1.0 +Content-Type: text/plain; charset=%(charset)s +Content-Transfer-Encoding: 8bit +Message-ID: %(msgid)s +From: %(fromaddr)s +Reply-To: %(reply_to)s +X-Git-Host: %(fqdn)s +X-Git-Repo: %(repo_shortname)s +X-Git-Refname: %(refname)s +X-Git-Reftype: %(refname_type)s +X-Git-Oldrev: %(oldrev)s +X-Git-Newrev: %(newrev)s +Auto-Submitted: auto-generated +""" + +REFCHANGE_INTRO_TEMPLATE = """\ +This is an automated email from the git hooks/post-receive script. + +%(pusher)s pushed a change to %(refname_type)s %(short_refname)s +in repository %(repo_shortname)s. + +""" + + +FOOTER_TEMPLATE = """\ + +-- \n\ +To stop receiving notification emails like this one, please contact +%(administrator)s. +""" + + +REWIND_ONLY_TEMPLATE = """\ +This update removed existing revisions from the reference, leaving the +reference pointing at a previous point in the repository history. + + * -- * -- N %(refname)s (%(newrev_short)s) + \\ + O -- O -- O (%(oldrev_short)s) + +Any revisions marked "omits" are not gone; other references still +refer to them. Any revisions marked "discards" are gone forever. +""" + + +NON_FF_TEMPLATE = """\ +This update added new revisions after undoing existing revisions. +That is to say, some revisions that were in the old version of the +%(refname_type)s are not in the new version. This situation occurs +when a user --force pushes a change and generates a repository +containing something like this: + + * -- * -- B -- O -- O -- O (%(oldrev_short)s) + \\ + N -- N -- N %(refname)s (%(newrev_short)s) + +You should already have received notification emails for all of the O +revisions, and so the following emails describe only the N revisions +from the common base, B. + +Any revisions marked "omits" are not gone; other references still +refer to them. Any revisions marked "discards" are gone forever. +""" + + +NO_NEW_REVISIONS_TEMPLATE = """\ +No new revisions were added by this update. +""" + + +DISCARDED_REVISIONS_TEMPLATE = """\ +This change permanently discards the following revisions: +""" + + +NO_DISCARDED_REVISIONS_TEMPLATE = """\ +The revisions that were on this %(refname_type)s are still contained in +other references; therefore, this change does not discard any commits +from the repository. +""" + + +NEW_REVISIONS_TEMPLATE = """\ +The %(tot)s revisions listed above as "new" are entirely new to this +repository and will be described in separate emails. The revisions +listed as "adds" were already present in the repository and have only +been added to this reference. + +""" + + +TAG_CREATED_TEMPLATE = """\ + at %(newrev_short)-9s (%(newrev_type)s) +""" + + +TAG_UPDATED_TEMPLATE = """\ +*** WARNING: tag %(short_refname)s was modified! *** + + from %(oldrev_short)-9s (%(oldrev_type)s) + to %(newrev_short)-9s (%(newrev_type)s) +""" + + +TAG_DELETED_TEMPLATE = """\ +*** WARNING: tag %(short_refname)s was deleted! *** + +""" + + +# The template used in summary tables. It looks best if this uses the +# same alignment as TAG_CREATED_TEMPLATE and TAG_UPDATED_TEMPLATE. +BRIEF_SUMMARY_TEMPLATE = """\ +%(action)10s %(rev_short)-9s %(text)s +""" + + +NON_COMMIT_UPDATE_TEMPLATE = """\ +This is an unusual reference change because the reference did not +refer to a commit either before or after the change. We do not know +how to provide full information about this reference change. +""" + + +REVISION_HEADER_TEMPLATE = """\ +Date: %(send_date)s +To: %(recipients)s +Cc: %(cc_recipients)s +Subject: %(emailprefix)s%(num)02d/%(tot)02d: %(oneline)s +MIME-Version: 1.0 +Content-Type: text/plain; charset=%(charset)s +Content-Transfer-Encoding: 8bit +From: %(fromaddr)s +Reply-To: %(reply_to)s +In-Reply-To: %(reply_to_msgid)s +References: %(reply_to_msgid)s +X-Git-Host: %(fqdn)s +X-Git-Repo: %(repo_shortname)s +X-Git-Refname: %(refname)s +X-Git-Reftype: %(refname_type)s +X-Git-Rev: %(rev)s +Auto-Submitted: auto-generated +""" + +REVISION_INTRO_TEMPLATE = """\ +This is an automated email from the git hooks/post-receive script. + +%(pusher)s pushed a commit to %(refname_type)s %(short_refname)s +in repository %(repo_shortname)s. + +""" + + +REVISION_FOOTER_TEMPLATE = FOOTER_TEMPLATE + + +# Combined, meaning refchange+revision email (for single-commit additions) +COMBINED_HEADER_TEMPLATE = """\ +Date: %(send_date)s +To: %(recipients)s +Subject: %(subject)s +MIME-Version: 1.0 +Content-Type: text/plain; charset=%(charset)s +Content-Transfer-Encoding: 8bit +Message-ID: %(msgid)s +From: %(fromaddr)s +Reply-To: %(reply_to)s +X-Git-Host: %(fqdn)s +X-Git-Repo: %(repo_shortname)s +X-Git-Refname: %(refname)s +X-Git-Reftype: %(refname_type)s +X-Git-Oldrev: %(oldrev)s +X-Git-Newrev: %(newrev)s +X-Git-Rev: %(rev)s +Auto-Submitted: auto-generated +""" + +COMBINED_INTRO_TEMPLATE = """\ +This is an automated email from the git hooks/post-receive script. + +%(pusher)s pushed a commit to %(refname_type)s %(short_refname)s +in repository %(repo_shortname)s. + +""" + +COMBINED_FOOTER_TEMPLATE = FOOTER_TEMPLATE + + +class CommandError(Exception): + def __init__(self, cmd, retcode): + self.cmd = cmd + self.retcode = retcode + Exception.__init__( + self, + 'Command "%s" failed with retcode %s' % (' '.join(cmd), retcode,) + ) + + +class ConfigurationException(Exception): + pass + + +# The "git" program (this could be changed to include a full path): +GIT_EXECUTABLE = 'git' + + +# How "git" should be invoked (including global arguments), as a list +# of words. This variable is usually initialized automatically by +# read_git_output() via choose_git_command(), but if a value is set +# here then it will be used unconditionally. +GIT_CMD = None + + +def choose_git_command(): + """Decide how to invoke git, and record the choice in GIT_CMD.""" + + global GIT_CMD + + if GIT_CMD is None: + try: + # Check to see whether the "-c" option is accepted (it was + # only added in Git 1.7.2). We don't actually use the + # output of "git --version", though if we needed more + # specific version information this would be the place to + # do it. + cmd = [GIT_EXECUTABLE, '-c', 'foo.bar=baz', '--version'] + read_output(cmd) + GIT_CMD = [GIT_EXECUTABLE, '-c', 'i18n.logoutputencoding=%s' % (ENCODING,)] + except CommandError: + GIT_CMD = [GIT_EXECUTABLE] + + +def read_git_output(args, input=None, keepends=False, **kw): + """Read the output of a Git command.""" + + if GIT_CMD is None: + choose_git_command() + + return read_output(GIT_CMD + args, input=input, keepends=keepends, **kw) + + +def read_output(cmd, input=None, keepends=False, **kw): + if input: + stdin = subprocess.PIPE + else: + stdin = None + p = subprocess.Popen( + cmd, stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kw + ) + (out, err) = p.communicate(input) + retcode = p.wait() + if retcode: + raise CommandError(cmd, retcode) + if not keepends: + out = out.rstrip('\n\r') + return out + + +def read_git_lines(args, keepends=False, **kw): + """Return the lines output by Git command. + + Return as single lines, with newlines stripped off.""" + + return read_git_output(args, keepends=True, **kw).splitlines(keepends) + + +def git_rev_list_ish(cmd, spec, args=None, **kw): + """Common functionality for invoking a 'git rev-list'-like command. + + Parameters: + * cmd is the Git command to run, e.g., 'rev-list' or 'log'. + * spec is a list of revision arguments to pass to the named + command. If None, this function returns an empty list. + * args is a list of extra arguments passed to the named command. + * All other keyword arguments (if any) are passed to the + underlying read_git_lines() function. + + Return the output of the Git command in the form of a list, one + entry per output line. + """ + if spec is None: + return [] + if args is None: + args = [] + args = [cmd, '--stdin'] + args + spec_stdin = ''.join(s + '\n' for s in spec) + return read_git_lines(args, input=spec_stdin, **kw) + + +def git_rev_list(spec, **kw): + """Run 'git rev-list' with the given list of revision arguments. + + See git_rev_list_ish() for parameter and return value + documentation. + """ + return git_rev_list_ish('rev-list', spec, **kw) + + +def git_log(spec, **kw): + """Run 'git log' with the given list of revision arguments. + + See git_rev_list_ish() for parameter and return value + documentation. + """ + return git_rev_list_ish('log', spec, **kw) + + +def header_encode(text, header_name=None): + """Encode and line-wrap the value of an email header field.""" + + try: + if isinstance(text, str): + text = text.decode(ENCODING, 'replace') + return Header(text, header_name=header_name).encode() + except UnicodeEncodeError: + return Header(text, header_name=header_name, charset=CHARSET, + errors='replace').encode() + + +def addr_header_encode(text, header_name=None): + """Encode and line-wrap the value of an email header field containing + email addresses.""" + + return Header( + ', '.join( + formataddr((header_encode(name), emailaddr)) + for name, emailaddr in getaddresses([text]) + ), + header_name=header_name + ).encode() + + +class Config(object): + def __init__(self, section, git_config=None): + """Represent a section of the git configuration. + + If git_config is specified, it is passed to "git config" in + the GIT_CONFIG environment variable, meaning that "git config" + will read the specified path rather than the Git default + config paths.""" + + self.section = section + if git_config: + self.env = os.environ.copy() + self.env['GIT_CONFIG'] = git_config + else: + self.env = None + + @staticmethod + def _split(s): + """Split NUL-terminated values.""" + + words = s.split('\0') + assert words[-1] == '' + return words[:-1] + + def get(self, name, default=None): + try: + values = self._split(read_git_output( + ['config', '--get', '--null', '%s.%s' % (self.section, name)], + env=self.env, keepends=True, + )) + assert len(values) == 1 + return values[0] + except CommandError: + return default + + def get_bool(self, name, default=None): + try: + value = read_git_output( + ['config', '--get', '--bool', '%s.%s' % (self.section, name)], + env=self.env, + ) + except CommandError: + return default + return value == 'true' + + def get_all(self, name, default=None): + """Read a (possibly multivalued) setting from the configuration. + + Return the result as a list of values, or default if the name + is unset.""" + + try: + return self._split(read_git_output( + ['config', '--get-all', '--null', '%s.%s' % (self.section, name)], + env=self.env, keepends=True, + )) + except CommandError, e: + if e.retcode == 1: + # "the section or key is invalid"; i.e., there is no + # value for the specified key. + return default + else: + raise + + def get_recipients(self, name, default=None): + """Read a recipients list from the configuration. + + Return the result as a comma-separated list of email + addresses, or default if the option is unset. If the setting + has multiple values, concatenate them with comma separators.""" + + lines = self.get_all(name, default=None) + if lines is None: + return default + return ', '.join(line.strip() for line in lines) + + def set(self, name, value): + read_git_output( + ['config', '%s.%s' % (self.section, name), value], + env=self.env, + ) + + def add(self, name, value): + read_git_output( + ['config', '--add', '%s.%s' % (self.section, name), value], + env=self.env, + ) + + def __contains__(self, name): + return self.get_all(name, default=None) is not None + + # We don't use this method anymore internally, but keep it here in + # case somebody is calling it from their own code: + def has_key(self, name): + return name in self + + def unset_all(self, name): + try: + read_git_output( + ['config', '--unset-all', '%s.%s' % (self.section, name)], + env=self.env, + ) + except CommandError, e: + if e.retcode == 5: + # The name doesn't exist, which is what we wanted anyway... + pass + else: + raise + + def set_recipients(self, name, value): + self.unset_all(name) + for pair in getaddresses([value]): + self.add(name, formataddr(pair)) + + +def generate_summaries(*log_args): + """Generate a brief summary for each revision requested. + + log_args are strings that will be passed directly to "git log" as + revision selectors. Iterate over (sha1_short, subject) for each + commit specified by log_args (subject is the first line of the + commit message as a string without EOLs).""" + + cmd = [ + 'log', '--abbrev', '--format=%h %s', + ] + list(log_args) + ['--'] + for line in read_git_lines(cmd): + yield tuple(line.split(' ', 1)) + + +def limit_lines(lines, max_lines): + for (index, line) in enumerate(lines): + if index < max_lines: + yield line + + if index >= max_lines: + yield '... %d lines suppressed ...\n' % (index + 1 - max_lines,) + + +def limit_linelength(lines, max_linelength): + for line in lines: + # Don't forget that lines always include a trailing newline. + if len(line) > max_linelength + 1: + line = line[:max_linelength - 7] + ' [...]\n' + yield line + + +class CommitSet(object): + """A (constant) set of object names. + + The set should be initialized with full SHA1 object names. The + __contains__() method returns True iff its argument is an + abbreviation of any the names in the set.""" + + def __init__(self, names): + self._names = sorted(names) + + def __len__(self): + return len(self._names) + + def __contains__(self, sha1_abbrev): + """Return True iff this set contains sha1_abbrev (which might be abbreviated).""" + + i = bisect.bisect_left(self._names, sha1_abbrev) + return i < len(self) and self._names[i].startswith(sha1_abbrev) + + +class GitObject(object): + def __init__(self, sha1, type=None): + if sha1 == ZEROS: + self.sha1 = self.type = self.commit_sha1 = None + else: + self.sha1 = sha1 + self.type = type or read_git_output(['cat-file', '-t', self.sha1]) + + if self.type == 'commit': + self.commit_sha1 = self.sha1 + elif self.type == 'tag': + try: + self.commit_sha1 = read_git_output( + ['rev-parse', '--verify', '%s^0' % (self.sha1,)] + ) + except CommandError: + # Cannot deref tag to determine commit_sha1 + self.commit_sha1 = None + else: + self.commit_sha1 = None + + self.short = read_git_output(['rev-parse', '--short', sha1]) + + def get_summary(self): + """Return (sha1_short, subject) for this commit.""" + + if not self.sha1: + raise ValueError('Empty commit has no summary') + + return iter(generate_summaries('--no-walk', self.sha1)).next() + + def __eq__(self, other): + return isinstance(other, GitObject) and self.sha1 == other.sha1 + + def __hash__(self): + return hash(self.sha1) + + def __nonzero__(self): + return bool(self.sha1) + + def __str__(self): + return self.sha1 or ZEROS + + +class Change(object): + """A Change that has been made to the Git repository. + + Abstract class from which both Revisions and ReferenceChanges are + derived. A Change knows how to generate a notification email + describing itself.""" + + def __init__(self, environment): + self.environment = environment + self._values = None + + def _compute_values(self): + """Return a dictionary {keyword: expansion} for this Change. + + Derived classes overload this method to add more entries to + the return value. This method is used internally by + get_values(). The return value should always be a new + dictionary.""" + + return self.environment.get_values() + + def get_values(self, **extra_values): + """Return a dictionary {keyword: expansion} for this Change. + + Return a dictionary mapping keywords to the values that they + should be expanded to for this Change (used when interpolating + template strings). If any keyword arguments are supplied, add + those to the return value as well. The return value is always + a new dictionary.""" + + if self._values is None: + self._values = self._compute_values() + + values = self._values.copy() + if extra_values: + values.update(extra_values) + return values + + def expand(self, template, **extra_values): + """Expand template. + + Expand the template (which should be a string) using string + interpolation of the values for this Change. If any keyword + arguments are provided, also include those in the keywords + available for interpolation.""" + + return template % self.get_values(**extra_values) + + def expand_lines(self, template, **extra_values): + """Break template into lines and expand each line.""" + + values = self.get_values(**extra_values) + for line in template.splitlines(True): + yield line % values + + def expand_header_lines(self, template, **extra_values): + """Break template into lines and expand each line as an RFC 2822 header. + + Encode values and split up lines that are too long. Silently + skip lines that contain references to unknown variables.""" + + values = self.get_values(**extra_values) + for line in template.splitlines(): + (name, value) = line.split(':', 1) + + try: + value = value % values + except KeyError, e: + if DEBUG: + self.environment.log_warning( + 'Warning: unknown variable %r in the following line; line skipped:\n' + ' %s\n' + % (e.args[0], line,) + ) + else: + if name.lower() in ADDR_HEADERS: + value = addr_header_encode(value, name) + else: + value = header_encode(value, name) + for splitline in ('%s: %s\n' % (name, value)).splitlines(True): + yield splitline + + def generate_email_header(self): + """Generate the RFC 2822 email headers for this Change, a line at a time. + + The output should not include the trailing blank line.""" + + raise NotImplementedError() + + def generate_email_intro(self): + """Generate the email intro for this Change, a line at a time. + + The output will be used as the standard boilerplate at the top + of the email body.""" + + raise NotImplementedError() + + def generate_email_body(self): + """Generate the main part of the email body, a line at a time. + + The text in the body might be truncated after a specified + number of lines (see multimailhook.emailmaxlines).""" + + raise NotImplementedError() + + def generate_email_footer(self): + """Generate the footer of the email, a line at a time. + + The footer is always included, irrespective of + multimailhook.emailmaxlines.""" + + raise NotImplementedError() + + def generate_email(self, push, body_filter=None, extra_header_values={}): + """Generate an email describing this change. + + Iterate over the lines (including the header lines) of an + email describing this change. If body_filter is not None, + then use it to filter the lines that are intended for the + email body. + + The extra_header_values field is received as a dict and not as + **kwargs, to allow passing other keyword arguments in the + future (e.g. passing extra values to generate_email_intro()""" + + for line in self.generate_email_header(**extra_header_values): + yield line + yield '\n' + for line in self.generate_email_intro(): + yield line + + body = self.generate_email_body(push) + if body_filter is not None: + body = body_filter(body) + for line in body: + yield line + + for line in self.generate_email_footer(): + yield line + + +class Revision(Change): + """A Change consisting of a single git commit.""" + + CC_RE = re.compile(r'^\s*C[Cc]:\s*(?P<to>[^#]+@[^\s#]*)\s*(#.*)?$') + + def __init__(self, reference_change, rev, num, tot): + Change.__init__(self, reference_change.environment) + self.reference_change = reference_change + self.rev = rev + self.change_type = self.reference_change.change_type + self.refname = self.reference_change.refname + self.num = num + self.tot = tot + self.author = read_git_output(['log', '--no-walk', '--format=%aN <%aE>', self.rev.sha1]) + self.recipients = self.environment.get_revision_recipients(self) + + self.cc_recipients = '' + if self.environment.get_scancommitforcc(): + self.cc_recipients = ', '.join(to.strip() for to in self._cc_recipients()) + if self.cc_recipients: + self.environment.log_msg( + 'Add %s to CC for %s\n' % (self.cc_recipients, self.rev.sha1)) + + def _cc_recipients(self): + cc_recipients = [] + message = read_git_output(['log', '--no-walk', '--format=%b', self.rev.sha1]) + lines = message.strip().split('\n') + for line in lines: + m = re.match(self.CC_RE, line) + if m: + cc_recipients.append(m.group('to')) + + return cc_recipients + + def _compute_values(self): + values = Change._compute_values(self) + + oneline = read_git_output( + ['log', '--format=%s', '--no-walk', self.rev.sha1] + ) + + values['rev'] = self.rev.sha1 + values['rev_short'] = self.rev.short + values['change_type'] = self.change_type + values['refname'] = self.refname + values['short_refname'] = self.reference_change.short_refname + values['refname_type'] = self.reference_change.refname_type + values['reply_to_msgid'] = self.reference_change.msgid + values['num'] = self.num + values['tot'] = self.tot + values['recipients'] = self.recipients + if self.cc_recipients: + values['cc_recipients'] = self.cc_recipients + values['oneline'] = oneline + values['author'] = self.author + + reply_to = self.environment.get_reply_to_commit(self) + if reply_to: + values['reply_to'] = reply_to + + return values + + def generate_email_header(self, **extra_values): + for line in self.expand_header_lines( + REVISION_HEADER_TEMPLATE, **extra_values + ): + yield line + + def generate_email_intro(self): + for line in self.expand_lines(REVISION_INTRO_TEMPLATE): + yield line + + def generate_email_body(self, push): + """Show this revision.""" + + return read_git_lines( + ['log'] + self.environment.commitlogopts + ['-1', self.rev.sha1], + keepends=True, + ) + + def generate_email_footer(self): + return self.expand_lines(REVISION_FOOTER_TEMPLATE) + + +class ReferenceChange(Change): + """A Change to a Git reference. + + An abstract class representing a create, update, or delete of a + Git reference. Derived classes handle specific types of reference + (e.g., tags vs. branches). These classes generate the main + reference change email summarizing the reference change and + whether it caused any any commits to be added or removed. + + ReferenceChange objects are usually created using the static + create() method, which has the logic to decide which derived class + to instantiate.""" + + REF_RE = re.compile(r'^refs\/(?P<area>[^\/]+)\/(?P<shortname>.*)$') + + @staticmethod + def create(environment, oldrev, newrev, refname): + """Return a ReferenceChange object representing the change. + + Return an object that represents the type of change that is being + made. oldrev and newrev should be SHA1s or ZEROS.""" + + old = GitObject(oldrev) + new = GitObject(newrev) + rev = new or old + + # The revision type tells us what type the commit is, combined with + # the location of the ref we can decide between + # - working branch + # - tracking branch + # - unannotated tag + # - annotated tag + m = ReferenceChange.REF_RE.match(refname) + if m: + area = m.group('area') + short_refname = m.group('shortname') + else: + area = '' + short_refname = refname + + if rev.type == 'tag': + # Annotated tag: + klass = AnnotatedTagChange + elif rev.type == 'commit': + if area == 'tags': + # Non-annotated tag: + klass = NonAnnotatedTagChange + elif area == 'heads': + # Branch: + klass = BranchChange + elif area == 'remotes': + # Tracking branch: + environment.log_warning( + '*** Push-update of tracking branch %r\n' + '*** - incomplete email generated.\n' + % (refname,) + ) + klass = OtherReferenceChange + else: + # Some other reference namespace: + environment.log_warning( + '*** Push-update of strange reference %r\n' + '*** - incomplete email generated.\n' + % (refname,) + ) + klass = OtherReferenceChange + else: + # Anything else (is there anything else?) + environment.log_warning( + '*** Unknown type of update to %r (%s)\n' + '*** - incomplete email generated.\n' + % (refname, rev.type,) + ) + klass = OtherReferenceChange + + return klass( + environment, + refname=refname, short_refname=short_refname, + old=old, new=new, rev=rev, + ) + + def __init__(self, environment, refname, short_refname, old, new, rev): + Change.__init__(self, environment) + self.change_type = { + (False, True): 'create', + (True, True): 'update', + (True, False): 'delete', + }[bool(old), bool(new)] + self.refname = refname + self.short_refname = short_refname + self.old = old + self.new = new + self.rev = rev + self.msgid = make_msgid() + self.diffopts = environment.diffopts + self.graphopts = environment.graphopts + self.logopts = environment.logopts + self.commitlogopts = environment.commitlogopts + self.showgraph = environment.refchange_showgraph + self.showlog = environment.refchange_showlog + + self.header_template = REFCHANGE_HEADER_TEMPLATE + self.intro_template = REFCHANGE_INTRO_TEMPLATE + self.footer_template = FOOTER_TEMPLATE + + def _compute_values(self): + values = Change._compute_values(self) + + values['change_type'] = self.change_type + values['refname_type'] = self.refname_type + values['refname'] = self.refname + values['short_refname'] = self.short_refname + values['msgid'] = self.msgid + values['recipients'] = self.recipients + values['oldrev'] = str(self.old) + values['oldrev_short'] = self.old.short + values['newrev'] = str(self.new) + values['newrev_short'] = self.new.short + + if self.old: + values['oldrev_type'] = self.old.type + if self.new: + values['newrev_type'] = self.new.type + + reply_to = self.environment.get_reply_to_refchange(self) + if reply_to: + values['reply_to'] = reply_to + + return values + + def send_single_combined_email(self, known_added_sha1s): + """Determine if a combined refchange/revision email should be sent + + If there is only a single new (non-merge) commit added by a + change, it is useful to combine the ReferenceChange and + Revision emails into one. In such a case, return the single + revision; otherwise, return None. + + This method is overridden in BranchChange.""" + + return None + + def generate_combined_email(self, push, revision, body_filter=None, extra_header_values={}): + """Generate an email describing this change AND specified revision. + + Iterate over the lines (including the header lines) of an + email describing this change. If body_filter is not None, + then use it to filter the lines that are intended for the + email body. + + The extra_header_values field is received as a dict and not as + **kwargs, to allow passing other keyword arguments in the + future (e.g. passing extra values to generate_email_intro() + + This method is overridden in BranchChange.""" + + raise NotImplementedError + + def get_subject(self): + template = { + 'create': REF_CREATED_SUBJECT_TEMPLATE, + 'update': REF_UPDATED_SUBJECT_TEMPLATE, + 'delete': REF_DELETED_SUBJECT_TEMPLATE, + }[self.change_type] + return self.expand(template) + + def generate_email_header(self, **extra_values): + if 'subject' not in extra_values: + extra_values['subject'] = self.get_subject() + + for line in self.expand_header_lines( + self.header_template, **extra_values + ): + yield line + + def generate_email_intro(self): + for line in self.expand_lines(self.intro_template): + yield line + + def generate_email_body(self, push): + """Call the appropriate body-generation routine. + + Call one of generate_create_summary() / + generate_update_summary() / generate_delete_summary().""" + + change_summary = { + 'create': self.generate_create_summary, + 'delete': self.generate_delete_summary, + 'update': self.generate_update_summary, + }[self.change_type](push) + for line in change_summary: + yield line + + for line in self.generate_revision_change_summary(push): + yield line + + def generate_email_footer(self): + return self.expand_lines(self.footer_template) + + def generate_revision_change_graph(self, push): + if self.showgraph: + args = ['--graph'] + self.graphopts + for newold in ('new', 'old'): + has_newold = False + spec = push.get_commits_spec(newold, self) + for line in git_log(spec, args=args, keepends=True): + if not has_newold: + has_newold = True + yield '\n' + yield 'Graph of %s commits:\n\n' % ( + {'new': 'new', 'old': 'discarded'}[newold],) + yield ' ' + line + if has_newold: + yield '\n' + + def generate_revision_change_log(self, new_commits_list): + if self.showlog: + yield '\n' + yield 'Detailed log of new commits:\n\n' + for line in read_git_lines( + ['log', '--no-walk'] + + self.logopts + + new_commits_list + + ['--'], + keepends=True, + ): + yield line + + def generate_new_revision_summary(self, tot, new_commits_list, push): + for line in self.expand_lines(NEW_REVISIONS_TEMPLATE, tot=tot): + yield line + for line in self.generate_revision_change_graph(push): + yield line + for line in self.generate_revision_change_log(new_commits_list): + yield line + + def generate_revision_change_summary(self, push): + """Generate a summary of the revisions added/removed by this change.""" + + if self.new.commit_sha1 and not self.old.commit_sha1: + # A new reference was created. List the new revisions + # brought by the new reference (i.e., those revisions that + # were not in the repository before this reference + # change). + sha1s = list(push.get_new_commits(self)) + sha1s.reverse() + tot = len(sha1s) + new_revisions = [ + Revision(self, GitObject(sha1), num=i + 1, tot=tot) + for (i, sha1) in enumerate(sha1s) + ] + + if new_revisions: + yield self.expand('This %(refname_type)s includes the following new commits:\n') + yield '\n' + for r in new_revisions: + (sha1, subject) = r.rev.get_summary() + yield r.expand( + BRIEF_SUMMARY_TEMPLATE, action='new', text=subject, + ) + yield '\n' + for line in self.generate_new_revision_summary( + tot, [r.rev.sha1 for r in new_revisions], push): + yield line + else: + for line in self.expand_lines(NO_NEW_REVISIONS_TEMPLATE): + yield line + + elif self.new.commit_sha1 and self.old.commit_sha1: + # A reference was changed to point at a different commit. + # List the revisions that were removed and/or added *from + # that reference* by this reference change, along with a + # diff between the trees for its old and new values. + + # List of the revisions that were added to the branch by + # this update. Note this list can include revisions that + # have already had notification emails; we want such + # revisions in the summary even though we will not send + # new notification emails for them. + adds = list(generate_summaries( + '--topo-order', '--reverse', '%s..%s' + % (self.old.commit_sha1, self.new.commit_sha1,) + )) + + # List of the revisions that were removed from the branch + # by this update. This will be empty except for + # non-fast-forward updates. + discards = list(generate_summaries( + '%s..%s' % (self.new.commit_sha1, self.old.commit_sha1,) + )) + + if adds: + new_commits_list = push.get_new_commits(self) + else: + new_commits_list = [] + new_commits = CommitSet(new_commits_list) + + if discards: + discarded_commits = CommitSet(push.get_discarded_commits(self)) + else: + discarded_commits = CommitSet([]) + + if discards and adds: + for (sha1, subject) in discards: + if sha1 in discarded_commits: + action = 'discards' + else: + action = 'omits' + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action=action, + rev_short=sha1, text=subject, + ) + for (sha1, subject) in adds: + if sha1 in new_commits: + action = 'new' + else: + action = 'adds' + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action=action, + rev_short=sha1, text=subject, + ) + yield '\n' + for line in self.expand_lines(NON_FF_TEMPLATE): + yield line + + elif discards: + for (sha1, subject) in discards: + if sha1 in discarded_commits: + action = 'discards' + else: + action = 'omits' + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action=action, + rev_short=sha1, text=subject, + ) + yield '\n' + for line in self.expand_lines(REWIND_ONLY_TEMPLATE): + yield line + + elif adds: + (sha1, subject) = self.old.get_summary() + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action='from', + rev_short=sha1, text=subject, + ) + for (sha1, subject) in adds: + if sha1 in new_commits: + action = 'new' + else: + action = 'adds' + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action=action, + rev_short=sha1, text=subject, + ) + + yield '\n' + + if new_commits: + for line in self.generate_new_revision_summary( + len(new_commits), new_commits_list, push): + yield line + else: + for line in self.expand_lines(NO_NEW_REVISIONS_TEMPLATE): + yield line + for line in self.generate_revision_change_graph(push): + yield line + + # The diffstat is shown from the old revision to the new + # revision. This is to show the truth of what happened in + # this change. There's no point showing the stat from the + # base to the new revision because the base is effectively a + # random revision at this point - the user will be interested + # in what this revision changed - including the undoing of + # previous revisions in the case of non-fast-forward updates. + yield '\n' + yield 'Summary of changes:\n' + for line in read_git_lines( + ['diff-tree'] + + self.diffopts + + ['%s..%s' % (self.old.commit_sha1, self.new.commit_sha1,)], + keepends=True, + ): + yield line + + elif self.old.commit_sha1 and not self.new.commit_sha1: + # A reference was deleted. List the revisions that were + # removed from the repository by this reference change. + + sha1s = list(push.get_discarded_commits(self)) + tot = len(sha1s) + discarded_revisions = [ + Revision(self, GitObject(sha1), num=i + 1, tot=tot) + for (i, sha1) in enumerate(sha1s) + ] + + if discarded_revisions: + for line in self.expand_lines(DISCARDED_REVISIONS_TEMPLATE): + yield line + yield '\n' + for r in discarded_revisions: + (sha1, subject) = r.rev.get_summary() + yield r.expand( + BRIEF_SUMMARY_TEMPLATE, action='discards', text=subject, + ) + for line in self.generate_revision_change_graph(push): + yield line + else: + for line in self.expand_lines(NO_DISCARDED_REVISIONS_TEMPLATE): + yield line + + elif not self.old.commit_sha1 and not self.new.commit_sha1: + for line in self.expand_lines(NON_COMMIT_UPDATE_TEMPLATE): + yield line + + def generate_create_summary(self, push): + """Called for the creation of a reference.""" + + # This is a new reference and so oldrev is not valid + (sha1, subject) = self.new.get_summary() + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action='at', + rev_short=sha1, text=subject, + ) + yield '\n' + + def generate_update_summary(self, push): + """Called for the change of a pre-existing branch.""" + + return iter([]) + + def generate_delete_summary(self, push): + """Called for the deletion of any type of reference.""" + + (sha1, subject) = self.old.get_summary() + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action='was', + rev_short=sha1, text=subject, + ) + yield '\n' + + +class BranchChange(ReferenceChange): + refname_type = 'branch' + + def __init__(self, environment, refname, short_refname, old, new, rev): + ReferenceChange.__init__( + self, environment, + refname=refname, short_refname=short_refname, + old=old, new=new, rev=rev, + ) + self.recipients = environment.get_refchange_recipients(self) + self._single_revision = None + + def send_single_combined_email(self, known_added_sha1s): + if not self.environment.combine_when_single_commit: + return None + + # In the sadly-all-too-frequent usecase of people pushing only + # one of their commits at a time to a repository, users feel + # the reference change summary emails are noise rather than + # important signal. This is because, in this particular + # usecase, there is a reference change summary email for each + # new commit, and all these summaries do is point out that + # there is one new commit (which can readily be inferred by + # the existence of the individual revision email that is also + # sent). In such cases, our users prefer there to be a combined + # reference change summary/new revision email. + # + # So, if the change is an update and it doesn't discard any + # commits, and it adds exactly one non-merge commit (gerrit + # forces a workflow where every commit is individually merged + # and the git-multimail hook fired off for just this one + # change), then we send a combined refchange/revision email. + try: + # If this change is a reference update that doesn't discard + # any commits... + if self.change_type != 'update': + return None + + if read_git_lines( + ['merge-base', self.old.sha1, self.new.sha1] + ) != [self.old.sha1]: + return None + + # Check if this update introduced exactly one non-merge + # commit: + + def split_line(line): + """Split line into (sha1, [parent,...]).""" + + words = line.split() + return (words[0], words[1:]) + + # Get the new commits introduced by the push as a list of + # (sha1, [parent,...]) + new_commits = [ + split_line(line) + for line in read_git_lines( + [ + 'log', '-3', '--format=%H %P', + '%s..%s' % (self.old.sha1, self.new.sha1), + ] + ) + ] + + if not new_commits: + return None + + # If the newest commit is a merge, save it for a later check + # but otherwise ignore it + merge = None + tot = len(new_commits) + if len(new_commits[0][1]) > 1: + merge = new_commits[0][0] + del new_commits[0] + + # Our primary check: we can't combine if more than one commit + # is introduced. We also currently only combine if the new + # commit is a non-merge commit, though it may make sense to + # combine if it is a merge as well. + if not ( + len(new_commits) == 1 + and len(new_commits[0][1]) == 1 + and new_commits[0][0] in known_added_sha1s + ): + return None + + # We do not want to combine revision and refchange emails if + # those go to separate locations. + rev = Revision(self, GitObject(new_commits[0][0]), 1, tot) + if rev.recipients != self.recipients: + return None + + # We ignored the newest commit if it was just a merge of the one + # commit being introduced. But we don't want to ignore that + # merge commit it it involved conflict resolutions. Check that. + if merge and merge != read_git_output(['diff-tree', '--cc', merge]): + return None + + # We can combine the refchange and one new revision emails + # into one. Return the Revision that a combined email should + # be sent about. + return rev + except CommandError: + # Cannot determine number of commits in old..new or new..old; + # don't combine reference/revision emails: + return None + + def generate_combined_email(self, push, revision, body_filter=None, extra_header_values={}): + values = revision.get_values() + if extra_header_values: + values.update(extra_header_values) + if 'subject' not in extra_header_values: + values['subject'] = self.expand(COMBINED_REFCHANGE_REVISION_SUBJECT_TEMPLATE, **values) + + self._single_revision = revision + self.header_template = COMBINED_HEADER_TEMPLATE + self.intro_template = COMBINED_INTRO_TEMPLATE + self.footer_template = COMBINED_FOOTER_TEMPLATE + for line in self.generate_email(push, body_filter, values): + yield line + + def generate_email_body(self, push): + '''Call the appropriate body generation routine. + + If this is a combined refchange/revision email, the special logic + for handling this combined email comes from this function. For + other cases, we just use the normal handling.''' + + # If self._single_revision isn't set; don't override + if not self._single_revision: + for line in super(BranchChange, self).generate_email_body(push): + yield line + return + + # This is a combined refchange/revision email; we first provide + # some info from the refchange portion, and then call the revision + # generate_email_body function to handle the revision portion. + adds = list(generate_summaries( + '--topo-order', '--reverse', '%s..%s' + % (self.old.commit_sha1, self.new.commit_sha1,) + )) + + yield self.expand("The following commit(s) were added to %(refname)s by this push:\n") + for (sha1, subject) in adds: + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action='new', + rev_short=sha1, text=subject, + ) + + yield self._single_revision.rev.short + " is described below\n" + yield '\n' + + for line in self._single_revision.generate_email_body(push): + yield line + + +class AnnotatedTagChange(ReferenceChange): + refname_type = 'annotated tag' + + def __init__(self, environment, refname, short_refname, old, new, rev): + ReferenceChange.__init__( + self, environment, + refname=refname, short_refname=short_refname, + old=old, new=new, rev=rev, + ) + self.recipients = environment.get_announce_recipients(self) + self.show_shortlog = environment.announce_show_shortlog + + ANNOTATED_TAG_FORMAT = ( + '%(*objectname)\n' + '%(*objecttype)\n' + '%(taggername)\n' + '%(taggerdate)' + ) + + def describe_tag(self, push): + """Describe the new value of an annotated tag.""" + + # Use git for-each-ref to pull out the individual fields from + # the tag + [tagobject, tagtype, tagger, tagged] = read_git_lines( + ['for-each-ref', '--format=%s' % (self.ANNOTATED_TAG_FORMAT,), self.refname], + ) + + yield self.expand( + BRIEF_SUMMARY_TEMPLATE, action='tagging', + rev_short=tagobject, text='(%s)' % (tagtype,), + ) + if tagtype == 'commit': + # If the tagged object is a commit, then we assume this is a + # release, and so we calculate which tag this tag is + # replacing + try: + prevtag = read_git_output(['describe', '--abbrev=0', '%s^' % (self.new,)]) + except CommandError: + prevtag = None + if prevtag: + yield ' replaces %s\n' % (prevtag,) + else: + prevtag = None + yield ' length %s bytes\n' % (read_git_output(['cat-file', '-s', tagobject]),) + + yield ' tagged by %s\n' % (tagger,) + yield ' on %s\n' % (tagged,) + yield '\n' + + # Show the content of the tag message; this might contain a + # change log or release notes so is worth displaying. + yield LOGBEGIN + contents = list(read_git_lines(['cat-file', 'tag', self.new.sha1], keepends=True)) + contents = contents[contents.index('\n') + 1:] + if contents and contents[-1][-1:] != '\n': + contents.append('\n') + for line in contents: + yield line + + if self.show_shortlog and tagtype == 'commit': + # Only commit tags make sense to have rev-list operations + # performed on them + yield '\n' + if prevtag: + # Show changes since the previous release + revlist = read_git_output( + ['rev-list', '--pretty=short', '%s..%s' % (prevtag, self.new,)], + keepends=True, + ) + else: + # No previous tag, show all the changes since time + # began + revlist = read_git_output( + ['rev-list', '--pretty=short', '%s' % (self.new,)], + keepends=True, + ) + for line in read_git_lines(['shortlog'], input=revlist, keepends=True): + yield line + + yield LOGEND + yield '\n' + + def generate_create_summary(self, push): + """Called for the creation of an annotated tag.""" + + for line in self.expand_lines(TAG_CREATED_TEMPLATE): + yield line + + for line in self.describe_tag(push): + yield line + + def generate_update_summary(self, push): + """Called for the update of an annotated tag. + + This is probably a rare event and may not even be allowed.""" + + for line in self.expand_lines(TAG_UPDATED_TEMPLATE): + yield line + + for line in self.describe_tag(push): + yield line + + def generate_delete_summary(self, push): + """Called when a non-annotated reference is updated.""" + + for line in self.expand_lines(TAG_DELETED_TEMPLATE): + yield line + + yield self.expand(' tag was %(oldrev_short)s\n') + yield '\n' + + +class NonAnnotatedTagChange(ReferenceChange): + refname_type = 'tag' + + def __init__(self, environment, refname, short_refname, old, new, rev): + ReferenceChange.__init__( + self, environment, + refname=refname, short_refname=short_refname, + old=old, new=new, rev=rev, + ) + self.recipients = environment.get_refchange_recipients(self) + + def generate_create_summary(self, push): + """Called for the creation of an annotated tag.""" + + for line in self.expand_lines(TAG_CREATED_TEMPLATE): + yield line + + def generate_update_summary(self, push): + """Called when a non-annotated reference is updated.""" + + for line in self.expand_lines(TAG_UPDATED_TEMPLATE): + yield line + + def generate_delete_summary(self, push): + """Called when a non-annotated reference is updated.""" + + for line in self.expand_lines(TAG_DELETED_TEMPLATE): + yield line + + for line in ReferenceChange.generate_delete_summary(self, push): + yield line + + +class OtherReferenceChange(ReferenceChange): + refname_type = 'reference' + + def __init__(self, environment, refname, short_refname, old, new, rev): + # We use the full refname as short_refname, because otherwise + # the full name of the reference would not be obvious from the + # text of the email. + ReferenceChange.__init__( + self, environment, + refname=refname, short_refname=refname, + old=old, new=new, rev=rev, + ) + self.recipients = environment.get_refchange_recipients(self) + + +class Mailer(object): + """An object that can send emails.""" + + def send(self, lines, to_addrs): + """Send an email consisting of lines. + + lines must be an iterable over the lines constituting the + header and body of the email. to_addrs is a list of recipient + addresses (can be needed even if lines already contains a + "To:" field). It can be either a string (comma-separated list + of email addresses) or a Python list of individual email + addresses. + + """ + + raise NotImplementedError() + + +class SendMailer(Mailer): + """Send emails using 'sendmail -oi -t'.""" + + SENDMAIL_CANDIDATES = [ + '/usr/sbin/sendmail', + '/usr/lib/sendmail', + ] + + @staticmethod + def find_sendmail(): + for path in SendMailer.SENDMAIL_CANDIDATES: + if os.access(path, os.X_OK): + return path + else: + raise ConfigurationException( + 'No sendmail executable found. ' + 'Try setting multimailhook.sendmailCommand.' + ) + + def __init__(self, command=None, envelopesender=None): + """Construct a SendMailer instance. + + command should be the command and arguments used to invoke + sendmail, as a list of strings. If an envelopesender is + provided, it will also be passed to the command, via '-f + envelopesender'.""" + + if command: + self.command = command[:] + else: + self.command = [self.find_sendmail(), '-oi', '-t'] + + if envelopesender: + self.command.extend(['-f', envelopesender]) + + def send(self, lines, to_addrs): + try: + p = subprocess.Popen(self.command, stdin=subprocess.PIPE) + except OSError, e: + sys.stderr.write( + '*** Cannot execute command: %s\n' % ' '.join(self.command) + + '*** %s\n' % str(e) + + '*** Try setting multimailhook.mailer to "smtp"\n' + '*** to send emails without using the sendmail command.\n' + ) + sys.exit(1) + try: + p.stdin.writelines(lines) + except Exception, e: + sys.stderr.write( + '*** Error while generating commit email\n' + '*** - mail sending aborted.\n' + ) + try: + # subprocess.terminate() is not available in Python 2.4 + p.terminate() + except AttributeError: + pass + raise e + else: + p.stdin.close() + retcode = p.wait() + if retcode: + raise CommandError(self.command, retcode) + + +class SMTPMailer(Mailer): + """Send emails using Python's smtplib.""" + + def __init__(self, envelopesender, smtpserver, + smtpservertimeout=10.0, smtpserverdebuglevel=0, + smtpencryption='none', + smtpuser='', smtppass='', + ): + if not envelopesender: + sys.stderr.write( + 'fatal: git_multimail: cannot use SMTPMailer without a sender address.\n' + 'please set either multimailhook.envelopeSender or user.email\n' + ) + sys.exit(1) + if smtpencryption == 'ssl' and not (smtpuser and smtppass): + raise ConfigurationException( + 'Cannot use SMTPMailer with security option ssl ' + 'without options username and password.' + ) + self.envelopesender = envelopesender + self.smtpserver = smtpserver + self.smtpservertimeout = smtpservertimeout + self.smtpserverdebuglevel = smtpserverdebuglevel + self.security = smtpencryption + self.username = smtpuser + self.password = smtppass + try: + def call(klass, server, timeout): + try: + return klass(server, timeout=timeout) + except TypeError: + # Old Python versions do not have timeout= argument. + return klass(server) + if self.security == 'none': + self.smtp = call(smtplib.SMTP, self.smtpserver, timeout=self.smtpservertimeout) + elif self.security == 'ssl': + self.smtp = call(smtplib.SMTP_SSL, self.smtpserver, timeout=self.smtpservertimeout) + elif self.security == 'tls': + if ':' not in self.smtpserver: + self.smtpserver += ':587' # default port for TLS + self.smtp = call(smtplib.SMTP, self.smtpserver, timeout=self.smtpservertimeout) + self.smtp.ehlo() + self.smtp.starttls() + self.smtp.ehlo() + else: + sys.stdout.write('*** Error: Control reached an invalid option. ***') + sys.exit(1) + if self.smtpserverdebuglevel > 0: + sys.stdout.write( + "*** Setting debug on for SMTP server connection (%s) ***\n" + % self.smtpserverdebuglevel) + self.smtp.set_debuglevel(self.smtpserverdebuglevel) + except Exception, e: + sys.stderr.write( + '*** Error establishing SMTP connection to %s ***\n' + % self.smtpserver) + sys.stderr.write('*** %s\n' % str(e)) + sys.exit(1) + + def __del__(self): + if hasattr(self, 'smtp'): + self.smtp.quit() + + def send(self, lines, to_addrs): + try: + if self.username or self.password: + sys.stderr.write("*** Authenticating as %s ***\n" % self.username) + self.smtp.login(self.username, self.password) + msg = ''.join(lines) + # turn comma-separated list into Python list if needed. + if isinstance(to_addrs, basestring): + to_addrs = [email for (name, email) in getaddresses([to_addrs])] + self.smtp.sendmail(self.envelopesender, to_addrs, msg) + except Exception, e: + sys.stderr.write('*** Error sending email ***\n') + sys.stderr.write('*** %s\n' % str(e)) + self.smtp.quit() + sys.exit(1) + + +class OutputMailer(Mailer): + """Write emails to an output stream, bracketed by lines of '=' characters. + + This is intended for debugging purposes.""" + + SEPARATOR = '=' * 75 + '\n' + + def __init__(self, f): + self.f = f + + def send(self, lines, to_addrs): + self.f.write(self.SEPARATOR) + self.f.writelines(lines) + self.f.write(self.SEPARATOR) + + +def get_git_dir(): + """Determine GIT_DIR. + + Determine GIT_DIR either from the GIT_DIR environment variable or + from the working directory, using Git's usual rules.""" + + try: + return read_git_output(['rev-parse', '--git-dir']) + except CommandError: + sys.stderr.write('fatal: git_multimail: not in a git directory\n') + sys.exit(1) + + +class Environment(object): + """Describes the environment in which the push is occurring. + + An Environment object encapsulates information about the local + environment. For example, it knows how to determine: + + * the name of the repository to which the push occurred + + * what user did the push + + * what users want to be informed about various types of changes. + + An Environment object is expected to have the following methods: + + get_repo_shortname() + + Return a short name for the repository, for display + purposes. + + get_repo_path() + + Return the absolute path to the Git repository. + + get_emailprefix() + + Return a string that will be prefixed to every email's + subject. + + get_pusher() + + Return the username of the person who pushed the changes. + This value is used in the email body to indicate who + pushed the change. + + get_pusher_email() (may return None) + + Return the email address of the person who pushed the + changes. The value should be a single RFC 2822 email + address as a string; e.g., "Joe User <user@example.com>" + if available, otherwise "user@example.com". If set, the + value is used as the Reply-To address for refchange + emails. If it is impossible to determine the pusher's + email, this attribute should be set to None (in which case + no Reply-To header will be output). + + get_sender() + + Return the address to be used as the 'From' email address + in the email envelope. + + get_fromaddr() + + Return the 'From' email address used in the email 'From:' + headers. (May be a full RFC 2822 email address like 'Joe + User <user@example.com>'.) + + get_administrator() + + Return the name and/or email of the repository + administrator. This value is used in the footer as the + person to whom requests to be removed from the + notification list should be sent. Ideally, it should + include a valid email address. + + get_reply_to_refchange() + get_reply_to_commit() + + Return the address to use in the email "Reply-To" header, + as a string. These can be an RFC 2822 email address, or + None to omit the "Reply-To" header. + get_reply_to_refchange() is used for refchange emails; + get_reply_to_commit() is used for individual commit + emails. + + They should also define the following attributes: + + announce_show_shortlog (bool) + + True iff announce emails should include a shortlog. + + refchange_showgraph (bool) + + True iff refchanges emails should include a detailed graph. + + refchange_showlog (bool) + + True iff refchanges emails should include a detailed log. + + diffopts (list of strings) + + The options that should be passed to 'git diff' for the + summary email. The value should be a list of strings + representing words to be passed to the command. + + graphopts (list of strings) + + Analogous to diffopts, but contains options passed to + 'git log --graph' when generating the detailed graph for + a set of commits (see refchange_showgraph) + + logopts (list of strings) + + Analogous to diffopts, but contains options passed to + 'git log' when generating the detailed log for a set of + commits (see refchange_showlog) + + commitlogopts (list of strings) + + The options that should be passed to 'git log' for each + commit mail. The value should be a list of strings + representing words to be passed to the command. + + quiet (bool) + On success do not write to stderr + + stdout (bool) + Write email to stdout rather than emailing. Useful for debugging + + combine_when_single_commit (bool) + + True if a combined email should be produced when a single + new commit is pushed to a branch, False otherwise. + + """ + + REPO_NAME_RE = re.compile(r'^(?P<name>.+?)(?:\.git)$') + + def __init__(self, osenv=None): + self.osenv = osenv or os.environ + self.announce_show_shortlog = False + self.maxcommitemails = 500 + self.diffopts = ['--stat', '--summary', '--find-copies-harder'] + self.graphopts = ['--oneline', '--decorate'] + self.logopts = [] + self.refchange_showgraph = False + self.refchange_showlog = False + self.commitlogopts = ['-C', '--stat', '-p', '--cc'] + self.quiet = False + self.stdout = False + self.combine_when_single_commit = True + + self.COMPUTED_KEYS = [ + 'administrator', + 'charset', + 'emailprefix', + 'fromaddr', + 'pusher', + 'pusher_email', + 'repo_path', + 'repo_shortname', + 'sender', + ] + + self._values = None + + def get_repo_shortname(self): + """Use the last part of the repo path, with ".git" stripped off if present.""" + + basename = os.path.basename(os.path.abspath(self.get_repo_path())) + m = self.REPO_NAME_RE.match(basename) + if m: + return m.group('name') + else: + return basename + + def get_pusher(self): + raise NotImplementedError() + + def get_pusher_email(self): + return None + + def get_fromaddr(self): + config = Config('user') + fromname = config.get('name', default='') + fromemail = config.get('email', default='') + if fromemail: + return formataddr([fromname, fromemail]) + return self.get_sender() + + def get_administrator(self): + return 'the administrator of this repository' + + def get_emailprefix(self): + return '' + + def get_repo_path(self): + if read_git_output(['rev-parse', '--is-bare-repository']) == 'true': + path = get_git_dir() + else: + path = read_git_output(['rev-parse', '--show-toplevel']) + return os.path.abspath(path) + + def get_charset(self): + return CHARSET + + def get_values(self): + """Return a dictionary {keyword: expansion} for this Environment. + + This method is called by Change._compute_values(). The keys + in the returned dictionary are available to be used in any of + the templates. The dictionary is created by calling + self.get_NAME() for each of the attributes named in + COMPUTED_KEYS and recording those that do not return None. + The return value is always a new dictionary.""" + + if self._values is None: + values = {} + + for key in self.COMPUTED_KEYS: + value = getattr(self, 'get_%s' % (key,))() + if value is not None: + values[key] = value + + self._values = values + + return self._values.copy() + + def get_refchange_recipients(self, refchange): + """Return the recipients for notifications about refchange. + + Return the list of email addresses to which notifications + about the specified ReferenceChange should be sent.""" + + raise NotImplementedError() + + def get_announce_recipients(self, annotated_tag_change): + """Return the recipients for notifications about annotated_tag_change. + + Return the list of email addresses to which notifications + about the specified AnnotatedTagChange should be sent.""" + + raise NotImplementedError() + + def get_reply_to_refchange(self, refchange): + return self.get_pusher_email() + + def get_revision_recipients(self, revision): + """Return the recipients for messages about revision. + + Return the list of email addresses to which notifications + about the specified Revision should be sent. This method + could be overridden, for example, to take into account the + contents of the revision when deciding whom to notify about + it. For example, there could be a scheme for users to express + interest in particular files or subdirectories, and only + receive notification emails for revisions that affecting those + files.""" + + raise NotImplementedError() + + def get_reply_to_commit(self, revision): + return revision.author + + def filter_body(self, lines): + """Filter the lines intended for an email body. + + lines is an iterable over the lines that would go into the + email body. Filter it (e.g., limit the number of lines, the + line length, character set, etc.), returning another iterable. + See FilterLinesEnvironmentMixin and MaxlinesEnvironmentMixin + for classes implementing this functionality.""" + + return lines + + def log_msg(self, msg): + """Write the string msg on a log file or on stderr. + + Sends the text to stderr by default, override to change the behavior.""" + sys.stderr.write(msg) + + def log_warning(self, msg): + """Write the string msg on a log file or on stderr. + + Sends the text to stderr by default, override to change the behavior.""" + sys.stderr.write(msg) + + def log_error(self, msg): + """Write the string msg on a log file or on stderr. + + Sends the text to stderr by default, override to change the behavior.""" + sys.stderr.write(msg) + + +class ConfigEnvironmentMixin(Environment): + """A mixin that sets self.config to its constructor's config argument. + + This class's constructor consumes the "config" argument. + + Mixins that need to inspect the config should inherit from this + class (1) to make sure that "config" is still in the constructor + arguments with its own constructor runs and/or (2) to be sure that + self.config is set after construction.""" + + def __init__(self, config, **kw): + super(ConfigEnvironmentMixin, self).__init__(**kw) + self.config = config + + +class ConfigOptionsEnvironmentMixin(ConfigEnvironmentMixin): + """An Environment that reads most of its information from "git config".""" + + def __init__(self, config, **kw): + super(ConfigOptionsEnvironmentMixin, self).__init__( + config=config, **kw + ) + + for var, cfg in ( + ('announce_show_shortlog', 'announceshortlog'), + ('refchange_showgraph', 'refchangeShowGraph'), + ('refchange_showlog', 'refchangeshowlog'), + ('quiet', 'quiet'), + ('stdout', 'stdout'), + ): + val = config.get_bool(cfg) + if val is not None: + setattr(self, var, val) + + maxcommitemails = config.get('maxcommitemails') + if maxcommitemails is not None: + try: + self.maxcommitemails = int(maxcommitemails) + except ValueError: + self.log_warning( + '*** Malformed value for multimailhook.maxCommitEmails: %s\n' % maxcommitemails + + '*** Expected a number. Ignoring.\n' + ) + + diffopts = config.get('diffopts') + if diffopts is not None: + self.diffopts = shlex.split(diffopts) + + graphopts = config.get('graphOpts') + if graphopts is not None: + self.graphopts = shlex.split(graphopts) + + logopts = config.get('logopts') + if logopts is not None: + self.logopts = shlex.split(logopts) + + commitlogopts = config.get('commitlogopts') + if commitlogopts is not None: + self.commitlogopts = shlex.split(commitlogopts) + + reply_to = config.get('replyTo') + self.__reply_to_refchange = config.get('replyToRefchange', default=reply_to) + if ( + self.__reply_to_refchange is not None + and self.__reply_to_refchange.lower() == 'author' + ): + raise ConfigurationException( + '"author" is not an allowed setting for replyToRefchange' + ) + self.__reply_to_commit = config.get('replyToCommit', default=reply_to) + + combine = config.get_bool('combineWhenSingleCommit') + if combine is not None: + self.combine_when_single_commit = combine + + def get_administrator(self): + return ( + self.config.get('administrator') + or self.get_sender() + or super(ConfigOptionsEnvironmentMixin, self).get_administrator() + ) + + def get_repo_shortname(self): + return ( + self.config.get('reponame') + or super(ConfigOptionsEnvironmentMixin, self).get_repo_shortname() + ) + + def get_emailprefix(self): + emailprefix = self.config.get('emailprefix') + if emailprefix is not None: + emailprefix = emailprefix.strip() + if emailprefix: + return emailprefix + ' ' + else: + return '' + else: + return '[%s] ' % (self.get_repo_shortname(),) + + def get_sender(self): + return self.config.get('envelopesender') + + def get_fromaddr(self): + fromaddr = self.config.get('from') + if fromaddr: + return fromaddr + return super(ConfigOptionsEnvironmentMixin, self).get_fromaddr() + + def get_reply_to_refchange(self, refchange): + if self.__reply_to_refchange is None: + return super(ConfigOptionsEnvironmentMixin, self).get_reply_to_refchange(refchange) + elif self.__reply_to_refchange.lower() == 'pusher': + return self.get_pusher_email() + elif self.__reply_to_refchange.lower() == 'none': + return None + else: + return self.__reply_to_refchange + + def get_reply_to_commit(self, revision): + if self.__reply_to_commit is None: + return super(ConfigOptionsEnvironmentMixin, self).get_reply_to_commit(revision) + elif self.__reply_to_commit.lower() == 'author': + return revision.author + elif self.__reply_to_commit.lower() == 'pusher': + return self.get_pusher_email() + elif self.__reply_to_commit.lower() == 'none': + return None + else: + return self.__reply_to_commit + + def get_scancommitforcc(self): + return self.config.get('scancommitforcc') + + +class FilterLinesEnvironmentMixin(Environment): + """Handle encoding and maximum line length of body lines. + + emailmaxlinelength (int or None) + + The maximum length of any single line in the email body. + Longer lines are truncated at that length with ' [...]' + appended. + + strict_utf8 (bool) + + If this field is set to True, then the email body text is + expected to be UTF-8. Any invalid characters are + converted to U+FFFD, the Unicode replacement character + (encoded as UTF-8, of course). + + """ + + def __init__(self, strict_utf8=True, emailmaxlinelength=500, **kw): + super(FilterLinesEnvironmentMixin, self).__init__(**kw) + self.__strict_utf8 = strict_utf8 + self.__emailmaxlinelength = emailmaxlinelength + + def filter_body(self, lines): + lines = super(FilterLinesEnvironmentMixin, self).filter_body(lines) + if self.__strict_utf8: + lines = (line.decode(ENCODING, 'replace') for line in lines) + # Limit the line length in Unicode-space to avoid + # splitting characters: + if self.__emailmaxlinelength: + lines = limit_linelength(lines, self.__emailmaxlinelength) + lines = (line.encode(ENCODING, 'replace') for line in lines) + elif self.__emailmaxlinelength: + lines = limit_linelength(lines, self.__emailmaxlinelength) + + return lines + + +class ConfigFilterLinesEnvironmentMixin( + ConfigEnvironmentMixin, + FilterLinesEnvironmentMixin, + ): + """Handle encoding and maximum line length based on config.""" + + def __init__(self, config, **kw): + strict_utf8 = config.get_bool('emailstrictutf8', default=None) + if strict_utf8 is not None: + kw['strict_utf8'] = strict_utf8 + + emailmaxlinelength = config.get('emailmaxlinelength') + if emailmaxlinelength is not None: + kw['emailmaxlinelength'] = int(emailmaxlinelength) + + super(ConfigFilterLinesEnvironmentMixin, self).__init__( + config=config, **kw + ) + + +class MaxlinesEnvironmentMixin(Environment): + """Limit the email body to a specified number of lines.""" + + def __init__(self, emailmaxlines, **kw): + super(MaxlinesEnvironmentMixin, self).__init__(**kw) + self.__emailmaxlines = emailmaxlines + + def filter_body(self, lines): + lines = super(MaxlinesEnvironmentMixin, self).filter_body(lines) + if self.__emailmaxlines: + lines = limit_lines(lines, self.__emailmaxlines) + return lines + + +class ConfigMaxlinesEnvironmentMixin( + ConfigEnvironmentMixin, + MaxlinesEnvironmentMixin, + ): + """Limit the email body to the number of lines specified in config.""" + + def __init__(self, config, **kw): + emailmaxlines = int(config.get('emailmaxlines', default='0')) + super(ConfigMaxlinesEnvironmentMixin, self).__init__( + config=config, + emailmaxlines=emailmaxlines, + **kw + ) + + +class FQDNEnvironmentMixin(Environment): + """A mixin that sets the host's FQDN to its constructor argument.""" + + def __init__(self, fqdn, **kw): + super(FQDNEnvironmentMixin, self).__init__(**kw) + self.COMPUTED_KEYS += ['fqdn'] + self.__fqdn = fqdn + + def get_fqdn(self): + """Return the fully-qualified domain name for this host. + + Return None if it is unavailable or unwanted.""" + + return self.__fqdn + + +class ConfigFQDNEnvironmentMixin( + ConfigEnvironmentMixin, + FQDNEnvironmentMixin, + ): + """Read the FQDN from the config.""" + + def __init__(self, config, **kw): + fqdn = config.get('fqdn') + super(ConfigFQDNEnvironmentMixin, self).__init__( + config=config, + fqdn=fqdn, + **kw + ) + + +class ComputeFQDNEnvironmentMixin(FQDNEnvironmentMixin): + """Get the FQDN by calling socket.getfqdn().""" + + def __init__(self, **kw): + super(ComputeFQDNEnvironmentMixin, self).__init__( + fqdn=socket.getfqdn(), + **kw + ) + + +class PusherDomainEnvironmentMixin(ConfigEnvironmentMixin): + """Deduce pusher_email from pusher by appending an emaildomain.""" + + def __init__(self, **kw): + super(PusherDomainEnvironmentMixin, self).__init__(**kw) + self.__emaildomain = self.config.get('emaildomain') + + def get_pusher_email(self): + if self.__emaildomain: + # Derive the pusher's full email address in the default way: + return '%s@%s' % (self.get_pusher(), self.__emaildomain) + else: + return super(PusherDomainEnvironmentMixin, self).get_pusher_email() + + +class StaticRecipientsEnvironmentMixin(Environment): + """Set recipients statically based on constructor parameters.""" + + def __init__( + self, + refchange_recipients, announce_recipients, revision_recipients, scancommitforcc, + **kw + ): + super(StaticRecipientsEnvironmentMixin, self).__init__(**kw) + + # The recipients for various types of notification emails, as + # RFC 2822 email addresses separated by commas (or the empty + # string if no recipients are configured). Although there is + # a mechanism to choose the recipient lists based on on the + # actual *contents* of the change being reported, we only + # choose based on the *type* of the change. Therefore we can + # compute them once and for all: + if not (refchange_recipients + or announce_recipients + or revision_recipients + or scancommitforcc): + raise ConfigurationException('No email recipients configured!') + self.__refchange_recipients = refchange_recipients + self.__announce_recipients = announce_recipients + self.__revision_recipients = revision_recipients + + def get_refchange_recipients(self, refchange): + return self.__refchange_recipients + + def get_announce_recipients(self, annotated_tag_change): + return self.__announce_recipients + + def get_revision_recipients(self, revision): + return self.__revision_recipients + + +class ConfigRecipientsEnvironmentMixin( + ConfigEnvironmentMixin, + StaticRecipientsEnvironmentMixin + ): + """Determine recipients statically based on config.""" + + def __init__(self, config, **kw): + super(ConfigRecipientsEnvironmentMixin, self).__init__( + config=config, + refchange_recipients=self._get_recipients( + config, 'refchangelist', 'mailinglist', + ), + announce_recipients=self._get_recipients( + config, 'announcelist', 'refchangelist', 'mailinglist', + ), + revision_recipients=self._get_recipients( + config, 'commitlist', 'mailinglist', + ), + scancommitforcc=config.get('scancommitforcc'), + **kw + ) + + def _get_recipients(self, config, *names): + """Return the recipients for a particular type of message. + + Return the list of email addresses to which a particular type + of notification email should be sent, by looking at the config + value for "multimailhook.$name" for each of names. Use the + value from the first name that is configured. The return + value is a (possibly empty) string containing RFC 2822 email + addresses separated by commas. If no configuration could be + found, raise a ConfigurationException.""" + + for name in names: + retval = config.get_recipients(name) + if retval is not None: + return retval + else: + return '' + + +class ProjectdescEnvironmentMixin(Environment): + """Make a "projectdesc" value available for templates. + + By default, it is set to the first line of $GIT_DIR/description + (if that file is present and appears to be set meaningfully).""" + + def __init__(self, **kw): + super(ProjectdescEnvironmentMixin, self).__init__(**kw) + self.COMPUTED_KEYS += ['projectdesc'] + + def get_projectdesc(self): + """Return a one-line descripition of the project.""" + + git_dir = get_git_dir() + try: + projectdesc = open(os.path.join(git_dir, 'description')).readline().strip() + if projectdesc and not projectdesc.startswith('Unnamed repository'): + return projectdesc + except IOError: + pass + + return 'UNNAMED PROJECT' + + +class GenericEnvironmentMixin(Environment): + def get_pusher(self): + return self.osenv.get('USER', self.osenv.get('USERNAME', 'unknown user')) + + +class GenericEnvironment( + ProjectdescEnvironmentMixin, + ConfigMaxlinesEnvironmentMixin, + ComputeFQDNEnvironmentMixin, + ConfigFilterLinesEnvironmentMixin, + ConfigRecipientsEnvironmentMixin, + PusherDomainEnvironmentMixin, + ConfigOptionsEnvironmentMixin, + GenericEnvironmentMixin, + Environment, + ): + pass + + +class GitoliteEnvironmentMixin(Environment): + def get_repo_shortname(self): + # The gitolite environment variable $GL_REPO is a pretty good + # repo_shortname (though it's probably not as good as a value + # the user might have explicitly put in his config). + return ( + self.osenv.get('GL_REPO', None) + or super(GitoliteEnvironmentMixin, self).get_repo_shortname() + ) + + def get_pusher(self): + return self.osenv.get('GL_USER', 'unknown user') + + def get_fromaddr(self): + GL_USER = self.osenv.get('GL_USER') + if GL_USER is not None: + # Find the path to gitolite.conf. Note that gitolite v3 + # did away with the GL_ADMINDIR and GL_CONF environment + # variables (they are now hard-coded). + GL_ADMINDIR = self.osenv.get( + 'GL_ADMINDIR', + os.path.expanduser(os.path.join('~', '.gitolite'))) + GL_CONF = self.osenv.get( + 'GL_CONF', + os.path.join(GL_ADMINDIR, 'conf', 'gitolite.conf')) + if os.path.isfile(GL_CONF): + f = open(GL_CONF, 'rU') + try: + in_user_emails_section = False + re_template = r'^\s*#\s*{}\s*$' + re_begin, re_user, re_end = ( + re.compile(re_template.format(x)) + for x in ( + r'BEGIN\s+USER\s+EMAILS', + re.escape(GL_USER) + r'\s+(.*)', + r'END\s+USER\s+EMAILS', + )) + for l in f: + l = l.rstrip('\n') + if not in_user_emails_section: + if re_begin.match(l): + in_user_emails_section = True + continue + if re_end.match(l): + break + m = re_user.match(l) + if m: + return m.group(1) + finally: + f.close() + return super(GitoliteEnvironmentMixin, self).get_fromaddr() + + +class IncrementalDateTime(object): + """Simple wrapper to give incremental date/times. + + Each call will result in a date/time a second later than the + previous call. This can be used to falsify email headers, to + increase the likelihood that email clients sort the emails + correctly.""" + + def __init__(self): + self.time = time.time() + + def next(self): + formatted = formatdate(self.time, True) + self.time += 1 + return formatted + + +class GitoliteEnvironment( + ProjectdescEnvironmentMixin, + ConfigMaxlinesEnvironmentMixin, + ComputeFQDNEnvironmentMixin, + ConfigFilterLinesEnvironmentMixin, + ConfigRecipientsEnvironmentMixin, + PusherDomainEnvironmentMixin, + ConfigOptionsEnvironmentMixin, + GitoliteEnvironmentMixin, + Environment, + ): + pass + + +class Push(object): + """Represent an entire push (i.e., a group of ReferenceChanges). + + It is easy to figure out what commits were added to a *branch* by + a Reference change: + + git rev-list change.old..change.new + + or removed from a *branch*: + + git rev-list change.new..change.old + + But it is not quite so trivial to determine which entirely new + commits were added to the *repository* by a push and which old + commits were discarded by a push. A big part of the job of this + class is to figure out these things, and to make sure that new + commits are only detailed once even if they were added to multiple + references. + + The first step is to determine the "other" references--those + unaffected by the current push. They are computed by listing all + references then removing any affected by this push. The results + are stored in Push._other_ref_sha1s. + + The commits contained in the repository before this push were + + git rev-list other1 other2 other3 ... change1.old change2.old ... + + Where "changeN.old" is the old value of one of the references + affected by this push. + + The commits contained in the repository after this push are + + git rev-list other1 other2 other3 ... change1.new change2.new ... + + The commits added by this push are the difference between these + two sets, which can be written + + git rev-list \ + ^other1 ^other2 ... \ + ^change1.old ^change2.old ... \ + change1.new change2.new ... + + The commits removed by this push can be computed by + + git rev-list \ + ^other1 ^other2 ... \ + ^change1.new ^change2.new ... \ + change1.old change2.old ... + + The last point is that it is possible that other pushes are + occurring simultaneously to this one, so reference values can + change at any time. It is impossible to eliminate all race + conditions, but we reduce the window of time during which problems + can occur by translating reference names to SHA1s as soon as + possible and working with SHA1s thereafter (because SHA1s are + immutable).""" + + # A map {(changeclass, changetype): integer} specifying the order + # that reference changes will be processed if multiple reference + # changes are included in a single push. The order is significant + # mostly because new commit notifications are threaded together + # with the first reference change that includes the commit. The + # following order thus causes commits to be grouped with branch + # changes (as opposed to tag changes) if possible. + SORT_ORDER = dict( + (value, i) for (i, value) in enumerate([ + (BranchChange, 'update'), + (BranchChange, 'create'), + (AnnotatedTagChange, 'update'), + (AnnotatedTagChange, 'create'), + (NonAnnotatedTagChange, 'update'), + (NonAnnotatedTagChange, 'create'), + (BranchChange, 'delete'), + (AnnotatedTagChange, 'delete'), + (NonAnnotatedTagChange, 'delete'), + (OtherReferenceChange, 'update'), + (OtherReferenceChange, 'create'), + (OtherReferenceChange, 'delete'), + ]) + ) + + def __init__(self, changes, ignore_other_refs=False): + self.changes = sorted(changes, key=self._sort_key) + self.__other_ref_sha1s = None + self.__cached_commits_spec = {} + + if ignore_other_refs: + self.__other_ref_sha1s = set() + + @classmethod + def _sort_key(klass, change): + return (klass.SORT_ORDER[change.__class__, change.change_type], change.refname,) + + @property + def _other_ref_sha1s(self): + """The GitObjects referred to by references unaffected by this push. + """ + if self.__other_ref_sha1s is None: + # The refnames being changed by this push: + updated_refs = set( + change.refname + for change in self.changes + ) + + # The SHA-1s of commits referred to by all references in this + # repository *except* updated_refs: + sha1s = set() + fmt = ( + '%(objectname) %(objecttype) %(refname)\n' + '%(*objectname) %(*objecttype) %(refname)' + ) + for line in read_git_lines( + ['for-each-ref', '--format=%s' % (fmt,)]): + (sha1, type, name) = line.split(' ', 2) + if sha1 and type == 'commit' and name not in updated_refs: + sha1s.add(sha1) + + self.__other_ref_sha1s = sha1s + + return self.__other_ref_sha1s + + def _get_commits_spec_incl(self, new_or_old, reference_change=None): + """Get new or old SHA-1 from one or each of the changed refs. + + Return a list of SHA-1 commit identifier strings suitable as + arguments to 'git rev-list' (or 'git log' or ...). The + returned identifiers are either the old or new values from one + or all of the changed references, depending on the values of + new_or_old and reference_change. + + new_or_old is either the string 'new' or the string 'old'. If + 'new', the returned SHA-1 identifiers are the new values from + each changed reference. If 'old', the SHA-1 identifiers are + the old values from each changed reference. + + If reference_change is specified and not None, only the new or + old reference from the specified reference is included in the + return value. + + This function returns None if there are no matching revisions + (e.g., because a branch was deleted and new_or_old is 'new'). + """ + + if not reference_change: + incl_spec = sorted( + getattr(change, new_or_old).sha1 + for change in self.changes + if getattr(change, new_or_old) + ) + if not incl_spec: + incl_spec = None + elif not getattr(reference_change, new_or_old).commit_sha1: + incl_spec = None + else: + incl_spec = [getattr(reference_change, new_or_old).commit_sha1] + return incl_spec + + def _get_commits_spec_excl(self, new_or_old): + """Get exclusion revisions for determining new or discarded commits. + + Return a list of strings suitable as arguments to 'git + rev-list' (or 'git log' or ...) that will exclude all + commits that, depending on the value of new_or_old, were + either previously in the repository (useful for determining + which commits are new to the repository) or currently in the + repository (useful for determining which commits were + discarded from the repository). + + new_or_old is either the string 'new' or the string 'old'. If + 'new', the commits to be excluded are those that were in the + repository before the push. If 'old', the commits to be + excluded are those that are currently in the repository. """ + + old_or_new = {'old': 'new', 'new': 'old'}[new_or_old] + excl_revs = self._other_ref_sha1s.union( + getattr(change, old_or_new).sha1 + for change in self.changes + if getattr(change, old_or_new).type in ['commit', 'tag'] + ) + return ['^' + sha1 for sha1 in sorted(excl_revs)] + + def get_commits_spec(self, new_or_old, reference_change=None): + """Get rev-list arguments for added or discarded commits. + + Return a list of strings suitable as arguments to 'git + rev-list' (or 'git log' or ...) that select those commits + that, depending on the value of new_or_old, are either new to + the repository or were discarded from the repository. + + new_or_old is either the string 'new' or the string 'old'. If + 'new', the returned list is used to select commits that are + new to the repository. If 'old', the returned value is used + to select the commits that have been discarded from the + repository. + + If reference_change is specified and not None, the new or + discarded commits are limited to those that are reachable from + the new or old value of the specified reference. + + This function returns None if there are no added (or discarded) + revisions. + """ + key = (new_or_old, reference_change) + if key not in self.__cached_commits_spec: + ret = self._get_commits_spec_incl(new_or_old, reference_change) + if ret is not None: + ret.extend(self._get_commits_spec_excl(new_or_old)) + self.__cached_commits_spec[key] = ret + return self.__cached_commits_spec[key] + + def get_new_commits(self, reference_change=None): + """Return a list of commits added by this push. + + Return a list of the object names of commits that were added + by the part of this push represented by reference_change. If + reference_change is None, then return a list of *all* commits + added by this push.""" + + spec = self.get_commits_spec('new', reference_change) + return git_rev_list(spec) + + def get_discarded_commits(self, reference_change): + """Return a list of commits discarded by this push. + + Return a list of the object names of commits that were + entirely discarded from the repository by the part of this + push represented by reference_change.""" + + spec = self.get_commits_spec('old', reference_change) + return git_rev_list(spec) + + def send_emails(self, mailer, body_filter=None): + """Use send all of the notification emails needed for this push. + + Use send all of the notification emails (including reference + change emails and commit emails) needed for this push. Send + the emails using mailer. If body_filter is not None, then use + it to filter the lines that are intended for the email + body.""" + + # The sha1s of commits that were introduced by this push. + # They will be removed from this set as they are processed, to + # guarantee that one (and only one) email is generated for + # each new commit. + unhandled_sha1s = set(self.get_new_commits()) + send_date = IncrementalDateTime() + for change in self.changes: + sha1s = [] + for sha1 in reversed(list(self.get_new_commits(change))): + if sha1 in unhandled_sha1s: + sha1s.append(sha1) + unhandled_sha1s.remove(sha1) + + # Check if we've got anyone to send to + if not change.recipients: + change.environment.log_warning( + '*** no recipients configured so no email will be sent\n' + '*** for %r update %s->%s\n' + % (change.refname, change.old.sha1, change.new.sha1,) + ) + else: + if not change.environment.quiet: + change.environment.log_msg( + 'Sending notification emails to: %s\n' % (change.recipients,)) + extra_values = {'send_date': send_date.next()} + + rev = change.send_single_combined_email(sha1s) + if rev: + mailer.send( + change.generate_combined_email(self, rev, body_filter, extra_values), + rev.recipients, + ) + # This change is now fully handled; no need to handle + # individual revisions any further. + continue + else: + mailer.send( + change.generate_email(self, body_filter, extra_values), + change.recipients, + ) + + max_emails = change.environment.maxcommitemails + if max_emails and len(sha1s) > max_emails: + change.environment.log_warning( + '*** Too many new commits (%d), not sending commit emails.\n' % len(sha1s) + + '*** Try setting multimailhook.maxCommitEmails to a greater value\n' + + '*** Currently, multimailhook.maxCommitEmails=%d\n' % max_emails + ) + return + + for (num, sha1) in enumerate(sha1s): + rev = Revision(change, GitObject(sha1), num=num + 1, tot=len(sha1s)) + if not rev.recipients and rev.cc_recipients: + change.environment.log_msg('*** Replacing Cc: with To:\n') + rev.recipients = rev.cc_recipients + rev.cc_recipients = None + if rev.recipients: + extra_values = {'send_date': send_date.next()} + mailer.send( + rev.generate_email(self, body_filter, extra_values), + rev.recipients, + ) + + # Consistency check: + if unhandled_sha1s: + change.environment.log_error( + 'ERROR: No emails were sent for the following new commits:\n' + ' %s\n' + % ('\n '.join(sorted(unhandled_sha1s)),) + ) + + +def run_as_post_receive_hook(environment, mailer): + changes = [] + for line in sys.stdin: + (oldrev, newrev, refname) = line.strip().split(' ', 2) + changes.append( + ReferenceChange.create(environment, oldrev, newrev, refname) + ) + push = Push(changes) + push.send_emails(mailer, body_filter=environment.filter_body) + + +def run_as_update_hook(environment, mailer, refname, oldrev, newrev, force_send=False): + changes = [ + ReferenceChange.create( + environment, + read_git_output(['rev-parse', '--verify', oldrev]), + read_git_output(['rev-parse', '--verify', newrev]), + refname, + ), + ] + push = Push(changes, force_send) + push.send_emails(mailer, body_filter=environment.filter_body) + + +def choose_mailer(config, environment): + mailer = config.get('mailer', default='sendmail') + + if mailer == 'smtp': + smtpserver = config.get('smtpserver', default='localhost') + smtpservertimeout = float(config.get('smtpservertimeout', default=10.0)) + smtpserverdebuglevel = int(config.get('smtpserverdebuglevel', default=0)) + smtpencryption = config.get('smtpencryption', default='none') + smtpuser = config.get('smtpuser', default='') + smtppass = config.get('smtppass', default='') + mailer = SMTPMailer( + envelopesender=(environment.get_sender() or environment.get_fromaddr()), + smtpserver=smtpserver, smtpservertimeout=smtpservertimeout, + smtpserverdebuglevel=smtpserverdebuglevel, + smtpencryption=smtpencryption, + smtpuser=smtpuser, + smtppass=smtppass, + ) + elif mailer == 'sendmail': + command = config.get('sendmailcommand') + if command: + command = shlex.split(command) + mailer = SendMailer(command=command, envelopesender=environment.get_sender()) + else: + environment.log_error( + 'fatal: multimailhook.mailer is set to an incorrect value: "%s"\n' % mailer + + 'please use one of "smtp" or "sendmail".\n' + ) + sys.exit(1) + return mailer + + +KNOWN_ENVIRONMENTS = { + 'generic': GenericEnvironmentMixin, + 'gitolite': GitoliteEnvironmentMixin, + } + + +def choose_environment(config, osenv=None, env=None, recipients=None): + if not osenv: + osenv = os.environ + + environment_mixins = [ + ProjectdescEnvironmentMixin, + ConfigMaxlinesEnvironmentMixin, + ComputeFQDNEnvironmentMixin, + ConfigFilterLinesEnvironmentMixin, + PusherDomainEnvironmentMixin, + ConfigOptionsEnvironmentMixin, + ] + environment_kw = { + 'osenv': osenv, + 'config': config, + } + + if not env: + env = config.get('environment') + + if not env: + if 'GL_USER' in osenv and 'GL_REPO' in osenv: + env = 'gitolite' + else: + env = 'generic' + + environment_mixins.append(KNOWN_ENVIRONMENTS[env]) + + if recipients: + environment_mixins.insert(0, StaticRecipientsEnvironmentMixin) + environment_kw['refchange_recipients'] = recipients + environment_kw['announce_recipients'] = recipients + environment_kw['revision_recipients'] = recipients + environment_kw['scancommitforcc'] = config.get('scancommitforcc') + else: + environment_mixins.insert(0, ConfigRecipientsEnvironmentMixin) + + environment_klass = type( + 'EffectiveEnvironment', + tuple(environment_mixins) + (Environment,), + {}, + ) + return environment_klass(**environment_kw) + + +def main(args): + parser = optparse.OptionParser( + description=__doc__, + usage='%prog [OPTIONS]\n or: %prog [OPTIONS] REFNAME OLDREV NEWREV', + ) + + parser.add_option( + '--environment', '--env', action='store', type='choice', + choices=['generic', 'gitolite'], default=None, + help=( + 'Choose type of environment is in use. Default is taken from ' + 'multimailhook.environment if set; otherwise "generic".' + ), + ) + parser.add_option( + '--stdout', action='store_true', default=False, + help='Output emails to stdout rather than sending them.', + ) + parser.add_option( + '--recipients', action='store', default=None, + help='Set list of email recipients for all types of emails.', + ) + parser.add_option( + '--show-env', action='store_true', default=False, + help=( + 'Write to stderr the values determined for the environment ' + '(intended for debugging purposes).' + ), + ) + parser.add_option( + '--force-send', action='store_true', default=False, + help=( + 'Force sending refchange email when using as an update hook. ' + 'This is useful to work around the unreliable new commits ' + 'detection in this mode.' + ), + ) + + (options, args) = parser.parse_args(args) + + config = Config('multimailhook') + + try: + environment = choose_environment( + config, osenv=os.environ, + env=options.environment, + recipients=options.recipients, + ) + + if options.show_env: + sys.stderr.write('Environment values:\n') + for (k, v) in sorted(environment.get_values().items()): + sys.stderr.write(' %s : %r\n' % (k, v)) + sys.stderr.write('\n') + + if options.stdout or environment.stdout: + mailer = OutputMailer(sys.stdout) + else: + mailer = choose_mailer(config, environment) + + # Dual mode: if arguments were specified on the command line, run + # like an update hook; otherwise, run as a post-receive hook. + if args: + if len(args) != 3: + parser.error('Need zero or three non-option arguments') + (refname, oldrev, newrev) = args + run_as_update_hook(environment, mailer, refname, oldrev, newrev, options.force_send) + else: + run_as_post_receive_hook(environment, mailer) + except ConfigurationException, e: + sys.exit(str(e)) + + +if __name__ == '__main__': + main(sys.argv[1:]) diff --git a/contrib/hooks/multimail/migrate-mailhook-config b/contrib/hooks/multimail/migrate-mailhook-config new file mode 100755 index 0000000000..d0e9b39201 --- /dev/null +++ b/contrib/hooks/multimail/migrate-mailhook-config @@ -0,0 +1,271 @@ +#! /usr/bin/env python2 + +"""Migrate a post-receive-email configuration to be usable with git_multimail.py. + +See README.migrate-from-post-receive-email for more information. + +""" + +import sys +import optparse + +from git_multimail import CommandError +from git_multimail import Config +from git_multimail import read_output + + +OLD_NAMES = [ + 'mailinglist', + 'announcelist', + 'envelopesender', + 'emailprefix', + 'showrev', + 'emailmaxlines', + 'diffopts', + 'scancommitforcc', + ] + +NEW_NAMES = [ + 'environment', + 'reponame', + 'mailinglist', + 'refchangelist', + 'commitlist', + 'announcelist', + 'announceshortlog', + 'envelopesender', + 'administrator', + 'emailprefix', + 'emailmaxlines', + 'diffopts', + 'emaildomain', + 'scancommitforcc', + ] + + +INFO = """\ + +SUCCESS! + +Your post-receive-email configuration has been converted to +git-multimail format. Please see README and +README.migrate-from-post-receive-email to learn about other +git-multimail configuration possibilities. + +For example, git-multimail has the following new options with no +equivalent in post-receive-email. You might want to read about them +to see if they would be useful in your situation: + +""" + + +def _check_old_config_exists(old): + """Check that at least one old configuration value is set.""" + + for name in OLD_NAMES: + if name in old: + return True + + return False + + +def _check_new_config_clear(new): + """Check that none of the new configuration names are set.""" + + retval = True + for name in NEW_NAMES: + if name in new: + if retval: + sys.stderr.write('INFO: The following configuration values already exist:\n\n') + sys.stderr.write(' "%s.%s"\n' % (new.section, name)) + retval = False + + return retval + + +def erase_values(config, names): + for name in names: + if name in config: + try: + sys.stderr.write('...unsetting "%s.%s"\n' % (config.section, name)) + config.unset_all(name) + except CommandError: + sys.stderr.write( + '\nWARNING: could not unset "%s.%s". ' + 'Perhaps it is not set at the --local level?\n\n' + % (config.section, name) + ) + + +def is_section_empty(section, local): + """Return True iff the specified configuration section is empty. + + Iff local is True, use the --local option when invoking 'git + config'.""" + + if local: + local_option = ['--local'] + else: + local_option = [] + + try: + read_output( + ['git', 'config'] + + local_option + + ['--get-regexp', '^%s\.' % (section,)] + ) + except CommandError, e: + if e.retcode == 1: + # This means that no settings were found. + return True + else: + raise + else: + return False + + +def remove_section_if_empty(section): + """If the specified configuration section is empty, delete it.""" + + try: + empty = is_section_empty(section, local=True) + except CommandError: + # Older versions of git do not support the --local option, so + # if the first attempt fails, try without --local. + try: + empty = is_section_empty(section, local=False) + except CommandError: + sys.stderr.write( + '\nINFO: If configuration section "%s.*" is empty, you might want ' + 'to delete it.\n\n' + % (section,) + ) + return + + if empty: + sys.stderr.write('...removing section "%s.*"\n' % (section,)) + read_output(['git', 'config', '--remove-section', section]) + else: + sys.stderr.write( + '\nINFO: Configuration section "%s.*" still has contents. ' + 'It will not be deleted.\n\n' + % (section,) + ) + + +def migrate_config(strict=False, retain=False, overwrite=False): + old = Config('hooks') + new = Config('multimailhook') + if not _check_old_config_exists(old): + sys.exit( + 'Your repository has no post-receive-email configuration. ' + 'Nothing to do.' + ) + if not _check_new_config_clear(new): + if overwrite: + sys.stderr.write('\nWARNING: Erasing the above values...\n\n') + erase_values(new, NEW_NAMES) + else: + sys.exit( + '\nERROR: Refusing to overwrite existing values. Use the --overwrite\n' + 'option to continue anyway.' + ) + + name = 'showrev' + if name in old: + msg = 'git-multimail does not support "%s.%s"' % (old.section, name,) + if strict: + sys.exit( + 'ERROR: %s.\n' + 'Please unset that value then try again, or run without --strict.' + % (msg,) + ) + else: + sys.stderr.write('\nWARNING: %s (ignoring).\n\n' % (msg,)) + + for name in ['mailinglist', 'announcelist']: + if name in old: + sys.stderr.write( + '...copying "%s.%s" to "%s.%s"\n' % (old.section, name, new.section, name) + ) + new.set_recipients(name, old.get_recipients(name)) + + if strict: + sys.stderr.write( + '...setting "%s.commitlist" to the empty string\n' % (new.section,) + ) + new.set_recipients('commitlist', '') + sys.stderr.write( + '...setting "%s.announceshortlog" to "true"\n' % (new.section,) + ) + new.set('announceshortlog', 'true') + + for name in ['envelopesender', 'emailmaxlines', 'diffopts', 'scancommitforcc']: + if name in old: + sys.stderr.write( + '...copying "%s.%s" to "%s.%s"\n' % (old.section, name, new.section, name) + ) + new.set(name, old.get(name)) + + name = 'emailprefix' + if name in old: + sys.stderr.write( + '...copying "%s.%s" to "%s.%s"\n' % (old.section, name, new.section, name) + ) + new.set(name, old.get(name)) + elif strict: + sys.stderr.write( + '...setting "%s.%s" to "[SCM]" to preserve old subject lines\n' + % (new.section, name) + ) + new.set(name, '[SCM]') + + if not retain: + erase_values(old, OLD_NAMES) + remove_section_if_empty(old.section) + + sys.stderr.write(INFO) + for name in NEW_NAMES: + if name not in OLD_NAMES: + sys.stderr.write(' "%s.%s"\n' % (new.section, name,)) + sys.stderr.write('\n') + + +def main(args): + parser = optparse.OptionParser( + description=__doc__, + usage='%prog [OPTIONS]', + ) + + parser.add_option( + '--strict', action='store_true', default=False, + help=( + 'Slavishly configure git-multimail as closely as possible to ' + 'the post-receive-email configuration. Default is to turn ' + 'on some new features that have no equivalent in post-receive-email.' + ), + ) + parser.add_option( + '--retain', action='store_true', default=False, + help=( + 'Retain the post-receive-email configuration values. ' + 'Default is to delete them after the new values are set.' + ), + ) + parser.add_option( + '--overwrite', action='store_true', default=False, + help=( + 'Overwrite any existing git-multimail configuration settings. ' + 'Default is to abort if such settings already exist.' + ), + ) + + (options, args) = parser.parse_args(args) + + if args: + parser.error('Unexpected arguments: %s' % (' '.join(args),)) + + migrate_config(strict=options.strict, retain=options.retain, overwrite=options.overwrite) + + +main(sys.argv[1:]) diff --git a/contrib/hooks/multimail/post-receive.example b/contrib/hooks/multimail/post-receive.example new file mode 100755 index 0000000000..43f7b6b635 --- /dev/null +++ b/contrib/hooks/multimail/post-receive.example @@ -0,0 +1,95 @@ +#! /usr/bin/env python2 + +"""Example post-receive hook based on git-multimail. + +The simplest way to use git-multimail is to use the script +git_multimail.py directly as a post-receive hook, and to configure it +using Git's configuration files and command-line parameters. You can +also write your own Python wrapper for more advanced configurability, +using git_multimail.py as a Python module. + +This script is a simple example of such a post-receive hook. It is +intended to be customized before use; see the comments in the script +to help you get started. + +Using git-multimail as a Python module as done here provides more +flexibility. It has the following advantages: + +* The tool's behavior can be customized using arbitrary Python code, + without having to edit git_multimail.py. + +* Configuration settings can be read from other sources; for example, + user names and email addresses could be read from LDAP or from a + database. Or the settings can even be hardcoded in the importing + Python script, if this is preferred. + +This script is a very basic example of how to use git_multimail.py as +a module. The comments below explain some of the points at which the +script's behavior could be changed or customized. + +""" + +import sys +import os + +# If necessary, add the path to the directory containing +# git_multimail.py to the Python path as follows. (This is not +# necessary if git_multimail.py is in the same directory as this +# script): + +#LIBDIR = 'path/to/directory/containing/module' +#sys.path.insert(0, LIBDIR) + +import git_multimail + + +# It is possible to modify the output templates here; e.g.: + +#git_multimail.FOOTER_TEMPLATE = """\ +# +#-- \n\ +#This email was generated by the wonderful git-multimail tool. +#""" + + +# Specify which "git config" section contains the configuration for +# git-multimail: +config = git_multimail.Config('multimailhook') + + +# Select the type of environment: +try: + environment = git_multimail.GenericEnvironment(config=config) + #environment = git_multimail.GitoliteEnvironment(config=config) +except git_multimail.ConfigurationException, e: + sys.exit(str(e)) + + +# Choose the method of sending emails based on the git config: +mailer = git_multimail.choose_mailer(config, environment) + +# Alternatively, you may hardcode the mailer using code like one of +# the following: + +# Use "/usr/sbin/sendmail -oi -t" to send emails. The envelopesender +# argument is optional: +#mailer = git_multimail.SendMailer( +# command=['/usr/sbin/sendmail', '-oi', '-t'], +# envelopesender='git-repo@example.com', +# ) + +# Use Python's smtplib to send emails. Both arguments are required. +#mailer = git_multimail.SMTPMailer( +# envelopesender='git-repo@example.com', +# # The smtpserver argument can also include a port number; e.g., +# # smtpserver='mail.example.com:25' +# smtpserver='mail.example.com', +# ) + +# OutputMailer is intended only for testing; it writes the emails to +# the specified file stream. +#mailer = git_multimail.OutputMailer(sys.stdout) + + +# Read changes from stdin and send notification emails: +git_multimail.run_as_post_receive_hook(environment, mailer) diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email index 0e5b72d7f1..8747b84334 100755 --- a/contrib/hooks/post-receive-email +++ b/contrib/hooks/post-receive-email @@ -2,10 +2,19 @@ # # Copyright (c) 2007 Andy Parkins # -# An example hook script to mail out commit update information. This hook -# sends emails listing new revisions to the repository introduced by the -# change being reported. The rule is that (for branch updates) each commit -# will appear on one email and one email only. +# An example hook script to mail out commit update information. +# +# NOTE: This script is no longer under active development. There +# is another script, git-multimail, which is more capable and +# configurable and is largely backwards-compatible with this script; +# please see "contrib/hooks/multimail/". For instructions on how to +# migrate from post-receive-email to git-multimail, please see +# "README.migrate-from-post-receive-email" in that directory. +# +# This hook sends emails listing new revisions to the repository +# introduced by the change being reported. The rule is that (for +# branch updates) each commit will appear on one email and one email +# only. # # This hook is stored in the contrib/hooks directory. Your distribution # will have put this somewhere standard. You should make this script @@ -13,7 +22,6 @@ # For example, on debian the hook is stored in # /usr/share/git-core/contrib/hooks/post-receive-email: # -# chmod a+x post-receive-email # cd /path/to/your/repository.git # ln -sf /usr/share/git-core/contrib/hooks/post-receive-email hooks/post-receive # @@ -233,6 +241,9 @@ generate_email_header() cat <<-EOF To: $recipients Subject: ${emailprefix}$projectdesc $refname_type $short_refname ${change_type}d. $describe + MIME-Version: 1.0 + Content-Type: text/plain; charset=utf-8 + Content-Transfer-Encoding: 8bit X-Git-Refname: $refname X-Git-Reftype: $refname_type X-Git-Oldrev: $oldrev @@ -462,7 +473,7 @@ generate_delete_branch_email() echo " was $oldrev" echo "" echo $LOGBEGIN - git show -s --pretty=oneline $oldrev + git diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev echo $LOGEND } @@ -538,11 +549,11 @@ generate_atag_email() # performed on them if [ -n "$prevtag" ]; then # Show changes since the previous release - git rev-list --pretty=short "$prevtag..$newrev" | git shortlog + git shortlog "$prevtag..$newrev" else # No previous tag, show all the changes since time # began - git rev-list --pretty=short $newrev | git shortlog + git shortlog $newrev fi ;; *) @@ -562,7 +573,7 @@ generate_delete_atag_email() echo " was $oldrev" echo "" echo $LOGBEGIN - git show -s --pretty=oneline $oldrev + git diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev echo $LOGEND } @@ -608,7 +619,7 @@ generate_general_email() echo "" if [ "$newrev_type" = "commit" ]; then echo $LOGBEGIN - git show --no-color --root -s --pretty=medium $newrev + git diff-tree -s --always --encoding=UTF-8 --pretty=medium $newrev echo $LOGEND else # What can we do here? The tag marks an object that is not @@ -627,7 +638,7 @@ generate_delete_general_email() echo " was $oldrev" echo "" echo $LOGBEGIN - git show -s --pretty=oneline $oldrev + git diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev echo $LOGEND } diff --git a/contrib/hooks/pre-auto-gc-battery b/contrib/hooks/pre-auto-gc-battery index 1f914c94aa..6a2cdebdb7 100644..100755 --- a/contrib/hooks/pre-auto-gc-battery +++ b/contrib/hooks/pre-auto-gc-battery @@ -13,7 +13,6 @@ # For example, if the hook is stored in # /usr/share/git-core/contrib/hooks/pre-auto-gc-battery: # -# chmod a+x pre-auto-gc-battery # cd /path/to/your/repository.git # ln -sf /usr/share/git-core/contrib/hooks/pre-auto-gc-battery \ # hooks/pre-auto-gc @@ -34,7 +33,7 @@ elif grep -q "AC Power \+: 1" /proc/pmu/info 2>/dev/null then exit 0 elif test -x /usr/bin/pmset && /usr/bin/pmset -g batt | - grep -q "Currently drawing from 'AC Power'" + grep -q "drawing from 'AC Power'" then exit 0 fi diff --git a/contrib/hooks/setgitperms.perl b/contrib/hooks/setgitperms.perl index 2770a1b1d2..2770a1b1d2 100644..100755 --- a/contrib/hooks/setgitperms.perl +++ b/contrib/hooks/setgitperms.perl diff --git a/contrib/hooks/update-paranoid b/contrib/hooks/update-paranoid index d18b317b2f..d18b317b2f 100644..100755 --- a/contrib/hooks/update-paranoid +++ b/contrib/hooks/update-paranoid diff --git a/contrib/mw-to-git/.gitignore b/contrib/mw-to-git/.gitignore index b9196555e5..ae545b013d 100644 --- a/contrib/mw-to-git/.gitignore +++ b/contrib/mw-to-git/.gitignore @@ -1 +1,2 @@ git-remote-mediawiki +git-mw diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm new file mode 100644 index 0000000000..d13c4dfa7d --- /dev/null +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -0,0 +1,100 @@ +package Git::Mediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; +} + +1; # Famous last words diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424481..a4b6f7a2cd 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,18 +2,56 @@ # Copyright (C) 2013 # Matthieu Moy <Matthieu.Moy@imag.fr> # -## Build git-remote-mediawiki +# To build and test: +# +# make +# bin-wrapper/git mw preview Some_page.mw +# bin-wrapper/git clone mediawiki::http://example.com/wiki/ +# +# To install, run Git's toplevel 'make install' then run: +# +# make install +GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ +INSTALL = install + SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ + -s --no-print-directory instlibdir) +DESTDIR_SQ = $(subst ','\'',$(DESTDIR)) +INSTLIBDIR_SQ = $(subst ','\'',$(INSTLIBDIR)) all: build -build install clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ - $@-perl-script +test: all + $(MAKE) -C t + +check: perlcritic test + +install_pm: + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(INSTLIBDIR_SQ)/Git' + $(INSTALL) -m 644 $(GIT_MEDIAWIKI_PM) \ + '$(DESTDIR_SQ)$(INSTLIBDIR_SQ)/$(GIT_MEDIAWIKI_PM)' + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ + build-perl-script + +install: install_pm + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ + install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ + clean-perl-script + perlcritic: - perlcritic -2 *.perl + perlcritic -5 $(SCRIPT_PERL) + -perlcritic -2 $(SCRIPT_PERL) + +.PHONY: all test check install_pm install clean perlcritic diff --git a/contrib/mw-to-git/bin-wrapper/git b/contrib/mw-to-git/bin-wrapper/git new file mode 100755 index 0000000000..6663ae57e8 --- /dev/null +++ b/contrib/mw-to-git/bin-wrapper/git @@ -0,0 +1,14 @@ +#!/bin/sh + +# git executable wrapper script for Git-Mediawiki to run tests without +# installing all the scripts and perl packages. + +GIT_ROOT_DIR=../../.. +GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd ${GIT_ROOT_DIR} && pwd) + +GITPERLLIB="$GIT_EXEC_PATH"'/contrib/mw-to-git'"${GITPERLLIB:+:$GITPERLLIB}" +PATH="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$PATH" + +export GITPERLLIB PATH + +exec "${GIT_EXEC_PATH}/bin-wrappers/git" "$@" diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100755 index 0000000000..28df3ee321 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,368 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person <benoit.person@ensimag.imag.fr> +# Celestin Matte <celestin.matte@ensimag.imag.fr> +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use Git::Mediawiki qw(clean_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +# Global parameters +my $verbose = 0; +sub v_print { + if ($verbose) { + return print {*STDERR} @_; + } + return; +} + +# Preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +sub file { + $file_name = shift; + return $file_name; +} + +my %commands = ( + 'help' => + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload + }, \&preview_help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV-1) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}, + 'verbose|v' => \$verbose); + +# Launch command +&{$cmd->[0]}; + +############################# Preview Functions ################################ + +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r <remote name>] [--autoload|-a] + [--output|-o <output filename>] [--verbose|-v] + <blob> | <filename> + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's +upstream if --remote is not set. If that remote is not found or if it +is not a mediawiki, it lists all mediawiki remotes configured and asks +you to replay your command with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in +the current dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed +by the remote mediawiki and combined with a template retrieved from +the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it +in your default web browser if the option --autoload is present. + +OPTIONS: + -r <remote name>, --remote <remote name> + If the remote is a mediawiki, the template and the parse engine + used for the preview will be those of that remote. + If not, a list of valid remotes will be shown. + + -a, --autoload + Try to load the HTML output in a new tab (or new window) of your + default web browser. + + -o <output filename>, --output <output filename> + Change the HTML output filename. Default filename is based on the + input filename with its extension replaced by '.html'. + + -v, --verbose + Show more information on what's going on under the hood. +END + exit; +} + +sub preview { + my $wiki; + my ($remote_url, $wiki_page_name); + my ($new_content, $template); + my $file_content; + + if ($file_name eq EMPTY) { + die "Missing file argument, see `git mw help`\n"; + } + + v_print("### Selecting remote\n"); + if ($remote_name eq EMPTY) { + $remote_name = find_upstream_remote_name(); + if ($remote_name) { + $remote_url = mediawiki_remote_url_maybe($remote_name); + } + + if (! $remote_url) { + my @valid_remotes = find_mediawiki_remotes(); + + if ($#valid_remotes == 0) { + print {*STDERR} "No mediawiki remote in this repo. \n"; + exit 1; + } else { + my $remotes_list = join("\n\t", @valid_remotes); + print {*STDERR} <<"MESSAGE"; +There are multiple mediawiki remotes, which of: + ${remotes_list} +do you want ? Use the -r option to specify the remote. +MESSAGE + } + + exit 1; + } + } else { + if (!is_valid_remote($remote_name)) { + die "${remote_name} is not a remote\n"; + } + + $remote_url = mediawiki_remote_url_maybe($remote_name); + if (! $remote_url) { + die "${remote_name} is not a mediawiki remote\n"; + } + } + v_print("selected remote:\n\tname: ${remote_name}\n\turl: ${remote_url}\n"); + + $wiki = connect_maybe($wiki, $remote_name, $remote_url); + + # Read file content + if (! -e $file_name) { + $file_content = git_cmd_try { + Git::command('cat-file', 'blob', $file_name); } + "%s failed w/ code %d"; + + if ($file_name =~ /(.+):(.+)/) { + $file_name = $2; + } + } else { + open my $read_fh, "<", $file_name + or die "could not open ${file_name}: $!\n"; + $file_content = do { local $/ = undef; <$read_fh> }; + close $read_fh + or die "unable to close: $!\n"; + } + + v_print("### Retrieving template\n"); + ($wiki_page_name = clean_filename($file_name)) =~ s/\.[^.]+$//; + $template = get_template($remote_url, $wiki_page_name); + + v_print("### Parsing local content\n"); + $new_content = $wiki->api({ + action => 'parse', + text => $file_content, + title => $wiki_page_name + }, { + skip_encoding => 1 + }) or die "No response from remote mediawiki\n"; + $new_content = $new_content->{'parse'}->{'text'}->{'*'}; + + v_print("### Merging contents\n"); + if ($preview_file_name eq EMPTY) { + ($preview_file_name = $file_name) =~ s/\.[^.]+$/.html/; + } + open(my $save_fh, '>:encoding(UTF-8)', $preview_file_name) + or die "Could not open: $!\n"; + print {$save_fh} merge_contents($template, $new_content, $remote_url); + close($save_fh) + or die "Could not close: $!\n"; + + v_print("### Results\n"); + if ($autoload) { + v_print("Launching browser w/ file: ${preview_file_name}"); + system('git', 'web--browse', $preview_file_name); + } else { + print {*STDERR} "Preview file saved as: ${preview_file_name}\n"; + } + + exit; +} + +# uses global scope variable: $remote_name +sub merge_contents { + my $template = shift; + my $content = shift; + my $remote_url = shift; + my ($content_tree, $html_tree, $mw_content_text); + my $template_content_id = 'bodyContent'; + + $html_tree = HTML::TreeBuilder->new; + $html_tree->parse($template); + + $content_tree = HTML::TreeBuilder->new; + $content_tree->parse($content); + + $template_content_id = Git::config("remote.${remote_name}.mwIDcontent") + || $template_content_id; + v_print("Using '${template_content_id}' as the content ID\n"); + + $mw_content_text = $html_tree->look_down('id', $template_content_id); + if (!defined $mw_content_text) { + print {*STDERR} <<"CONFIG"; +Could not combine the new content with the template. You might want to +configure `mediawiki.IDContent` in your config: + git config --add remote.${remote_name}.mwIDcontent <id> +and re-run the command afterward. +CONFIG + exit 1; + } + $mw_content_text->delete_content(); + $mw_content_text->push_content($content_tree); + + make_links_absolute($html_tree, $remote_url); + + return $html_tree->as_HTML; +} + +sub make_links_absolute { + my $html_tree = shift; + my $remote_url = shift; + for (@{ $html_tree->extract_links() }) { + my ($link, $element, $attr) = @{ $_ }; + my $url = url($link)->canonical; + if ($url !~ /#/) { + $element->attr($attr, URI->new_abs($url, $remote_url)); + } + } + return $html_tree; +} + +sub is_valid_remote { + my $remote = shift; + my @remotes = git_cmd_try { + Git::command('remote') } + "%s failed w/ code %d"; + my $found_remote = 0; + foreach my $remote (@remotes) { + if ($remote eq $remote) { + $found_remote = 1; + last; + } + } + return $found_remote; +} + +sub find_mediawiki_remotes { + my @remotes = git_cmd_try { + Git::command('remote'); } + "%s failed w/ code %d"; + my $remote_url; + my @valid_remotes = (); + foreach my $remote (@remotes) { + $remote_url = mediawiki_remote_url_maybe($remote); + if ($remote_url) { + push(@valid_remotes, $remote); + } + } + return @valid_remotes; +} + +sub find_upstream_remote_name { + my $current_branch = git_cmd_try { + Git::command_oneline('symbolic-ref', '--short', 'HEAD') } + "%s failed w/ code %d"; + return Git::config("branch.${current_branch}.remote"); +} + +sub mediawiki_remote_url_maybe { + my $remote = shift; + + # Find remote url + my $remote_url = Git::config("remote.${remote}.url"); + if ($remote_url =~ s/mediawiki::(.*)/$1/) { + return url($remote_url)->canonical; + } + + return; +} + +sub get_template { + my $url = shift; + my $page_name = shift; + my ($req, $res, $code, $url_after); + + $req = LWP::UserAgent->new; + if ($verbose) { + $req->show_progress(1); + } + + $res = $req->get("${url}/index.php?title=${page_name}"); + if (!$res->is_success) { + $code = $res->code; + $url_after = $res->request()->uri(); # resolve all redirections + if ($code == HTTP_CODE_PAGE_NOT_FOUND) { + if ($verbose) { + print {*STDERR} <<"WARNING"; +Warning: Failed to retrieve '$page_name'. Create it on the mediawiki if you want +all the links to work properly. +Trying to use the mediawiki homepage as a fallback template ... +WARNING + } + + # LWP automatically redirects GET request + $res = $req->get("${url}/index.php"); + if (!$res->is_success) { + $url_after = $res->request()->uri(); # resolve all redirections + die "Failed to get homepage @ ${url_after} w/ code ${code}\n"; + } + } else { + die "Failed to get '${page_name}' @ ${url_after} w/ code ${code}\n"; + } + } + + return $res->decoded_content; +} + +############################## Help Functions ################################## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw <command> <args> + +git mw commands are: + help Display help information about git mw + preview Parse and render local file into HTML +END + exit; +} diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 71baf8ace8..8dd74a9a40 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)'; use URI::Escape; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => '%2F'; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are replaced with this content. use constant DELETED_CONTENT => "[[Category:Deleted]]\n"; @@ -40,8 +39,6 @@ use constant NULL_SHA1 => '0000000000000000000000000000000000000000'; # Used on Git's side to reflect empty edit messages on the wiki use constant EMPTY_MESSAGE => '*Empty MediaWiki Message*'; -use constant EMPTY => q{}; - # Number of pages taken into account at once in submodule get_mw_page_list use constant SLICE_SIZE => 50; @@ -50,8 +47,6 @@ use constant SLICE_SIZE => 50; # the number of links to be returned (500 links max). use constant BATCH_SIZE => 10; -use constant HTTP_CODE_OK => 200; - if (@ARGV != 2) { exit_error_usage(); } @@ -199,37 +194,6 @@ sub parse_command { # MediaWiki API instance, created lazily. my $mediawiki; -sub mw_connect_maybe { - if ($mediawiki) { - return; - } - $mediawiki = MediaWiki::API->new; - $mediawiki->{config}->{api_url} = "${url}/api.php"; - if ($wiki_login) { - my %credential = ( - 'url' => $url, - 'username' => $wiki_login, - 'password' => $wiki_passwd - ); - Git::credential(\%credential); - my $request = {lgname => $credential{username}, - lgpassword => $credential{password}, - lgdomain => $wiki_domain}; - if ($mediawiki->login($request)) { - Git::credential(\%credential, 'approve'); - print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); - } else { - print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${url}\n); - print {*STDERR} ' (error ' . - $mediawiki->{error}->{code} . ': ' . - $mediawiki->{error}->{details} . ")\n"; - Git::credential(\%credential, 'reject'); - exit 1; - } - } - return; -} - sub fatal_mw_error { my $action = shift; print STDERR "fatal: could not $action.\n"; @@ -339,7 +303,7 @@ sub get_mw_first_pages { # Get the list of pages to be fetched according to configuration. sub get_mw_pages { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); print {*STDERR} "Listing pages on remote wiki...\n"; @@ -497,7 +461,12 @@ sub download_mw_mediafile { my $response = $mediawiki->{ua}->get($download_url); if ($response->code == HTTP_CODE_OK) { - return $response->decoded_content; + # It is tempting to return + # $response->decoded_content({charset => "none"}), but + # when doing so, utf8::downgrade($content) fails with + # "Wide character in subroutine entry". + $response->decode(); + return $response->content(); } else { print {*STDERR} "Error downloading mediafile from :\n"; print {*STDERR} "URL: ${download_url}\n"; @@ -529,7 +498,7 @@ sub get_last_local_revision { # avoid a loop onto all tracked pages. This is useful for the fetch-by-rev # option. sub get_last_global_remote_rev { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); my $query = { action => 'query', @@ -545,7 +514,7 @@ sub get_last_global_remote_rev { # Get the last remote revision concerning the tracked pages and the tracked # categories. sub get_last_remote_revision { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); my %pages_hash = get_mw_pages(); my @pages = values(%pages_hash); @@ -601,29 +570,6 @@ sub mediawiki_smudge { return "${string}\n"; } -sub mediawiki_clean_filename { - my $filename = shift; - $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; - # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. - # Do a variant of URL-encoding, i.e. looks like URL-encoding, - # but with _ added to prevent MediaWiki from thinking this is - # an actual special character. - $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; - # If we use the uri escape before - # we should unescape here, before anything - - return $filename; -} - -sub mediawiki_smudge_filename { - my $filename = shift; - $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; - $filename =~ s/ /_/g; - # Decode forbidden characters encoded in mediawiki_clean_filename - $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; - return $filename; -} - sub literal_data { my ($content) = @_; print {*STDOUT} 'data ', bytes::length($content), "\n", $content; @@ -635,9 +581,9 @@ sub literal_data_raw { my ($content) = @_; # Avoid confusion between size in bytes and in characters utf8::downgrade($content); - binmode {*STDOUT}, ':raw'; + binmode STDOUT, ':raw'; print {*STDOUT} 'data ', bytes::length($content), "\n", $content; - binmode {*STDOUT}, ':encoding(UTF-8)'; + binmode STDOUT, ':encoding(UTF-8)'; return; } @@ -649,6 +595,9 @@ sub mw_capabilities { print {*STDOUT} "import\n"; print {*STDOUT} "list\n"; print {*STDOUT} "push\n"; + if ($dumb_push) { + print {*STDOUT} "no-private-update\n"; + } print {*STDOUT} "\n"; return; } @@ -681,6 +630,9 @@ sub fetch_mw_revisions_for_page { rvstartid => $fetch_from, rvlimit => 500, pageids => $id, + + # Let MediaWiki know that we support the latest API. + continue => '', }; my $revnum = 0; @@ -696,8 +648,15 @@ sub fetch_mw_revisions_for_page { push(@page_revs, $page_rev_ids); $revnum++; } - last if (!$result->{'query-continue'}); - $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + + if ($result->{'query-continue'}) { # For legacy APIs + $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + } elsif ($result->{continue}) { # For newer APIs + $query->{rvstartid} = $result->{continue}->{rvcontinue}; + $query->{continue} = $result->{continue}->{continue}; + } else { + last; + } } if ($shallow_import && @page_revs) { print {*STDERR} " Found 1 revision (shallow import).\n"; @@ -831,7 +790,7 @@ sub mw_import_ref { return; } - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); print {*STDERR} "Searching revisions...\n"; my $last_local = get_last_local_revision(); @@ -945,7 +904,7 @@ sub mw_import_revids { my %commit; $commit{author} = $rev->{user} || 'Anonymous'; $commit{comment} = $rev->{comment} || EMPTY_MESSAGE; - $commit{title} = mediawiki_smudge_filename($page_title); + $commit{title} = smudge_filename($page_title); $commit{mw_revision} = $rev->{revid}; $commit{content} = mediawiki_smudge($rev->{'*'}); @@ -1006,7 +965,7 @@ sub mw_upload_file { } # Deleting and uploading a file requires a priviledged user if ($file_deleted) { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); my $query = { action => 'delete', title => $path, @@ -1022,7 +981,7 @@ sub mw_upload_file { # Don't let perl try to interpret file content as UTF-8 => use "raw" my $content = run_git("cat-file blob ${new_sha1}", 'raw'); if ($content ne EMPTY) { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); $mediawiki->{config}->{upload_url} = "${url}/index.php/Special:Upload"; $mediawiki->edit({ @@ -1070,7 +1029,7 @@ sub mw_push_file { my $old_sha1 = $diff_info_split[2]; my $page_created = ($old_sha1 eq NULL_SHA1); my $page_deleted = ($new_sha1 eq NULL_SHA1); - $complete_file_name = mediawiki_clean_filename($complete_file_name); + $complete_file_name = clean_filename($complete_file_name); my ($title, $extension) = $complete_file_name =~ /^(.*)\.([^\.]*)$/; if (!defined($extension)) { @@ -1093,7 +1052,7 @@ sub mw_push_file { $file_content = run_git("cat-file blob ${new_sha1}"); } - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); my $result = $mediawiki->edit( { action => 'edit', @@ -1270,7 +1229,6 @@ sub mw_push_revision { } if (!$dumb_push) { run_git(qq(notes --ref=${remotename}/mediawiki add -f -m "mediawiki_revision: ${mw_revision}" ${sha1_commit})); - run_git(qq(update-ref -m "Git-MediaWiki push" refs/mediawiki/${remotename}/master ${sha1_commit} ${sha1_child})); } } @@ -1279,7 +1237,7 @@ sub mw_push_revision { } sub get_allowed_file_extensions { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); my $query = { action => 'query', @@ -1303,7 +1261,7 @@ my %cached_mw_namespace_id; # Return MediaWiki id for a canonical namespace name. # Ex.: "File", "Project". sub get_mw_namespace_id { - mw_connect_maybe(); + $mediawiki = connect_maybe($mediawiki, $remotename, $url); my $name = shift; if (!exists $namespace_id{$name}) { @@ -1362,7 +1320,7 @@ sub get_mw_namespace_id { # Store "notANameSpace" as special value for inexisting namespaces my $store_id = ($id || 'notANameSpace'); - # Store explicitely requested namespaces on disk + # Store explicitly requested namespaces on disk if (!exists $cached_mw_namespace_id{$name}) { run_git(qq(config --add remote.${remotename}.namespaceCache "${name}:${store_id}")); $cached_mw_namespace_id{$name} = 1; diff --git a/contrib/mw-to-git/t/install-wiki.sh b/contrib/mw-to-git/t/install-wiki.sh index 70a53f67fd..c215213c4b 100755 --- a/contrib/mw-to-git/t/install-wiki.sh +++ b/contrib/mw-to-git/t/install-wiki.sh @@ -20,6 +20,8 @@ usage () { echo " install | -i : Install a wiki on your computer." echo " delete | -d : Delete the wiki and all its pages and " echo " content." + echo " start | -s : Start the previously configured lighttpd daemon" + echo " stop : Stop lighttpd daemon." } @@ -33,6 +35,14 @@ case "$1" in wiki_delete exit 0 ;; + "start" | "-s") + start_lighttpd + exit + ;; + "stop") + stop_lighttpd + exit + ;; "--help" | "-h") usage exit 0 diff --git a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh index 811a90c9ae..22f069db48 100755 --- a/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh +++ b/contrib/mw-to-git/t/t9360-mw-to-git-clone.sh @@ -191,10 +191,10 @@ test_expect_success 'Git clone works with the shallow option' ' test_path_is_file mw_dir_11/Main_Page.mw && ( cd mw_dir_11 && - test `git log --oneline Nyan.mw | wc -l` -eq 1 && - test `git log --oneline Foo.mw | wc -l` -eq 1 && - test `git log --oneline Bar.mw | wc -l` -eq 1 && - test `git log --oneline Main_Page.mw | wc -l ` -eq 1 + test $(git log --oneline Nyan.mw | wc -l) -eq 1 && + test $(git log --oneline Foo.mw | wc -l) -eq 1 && + test $(git log --oneline Bar.mw | wc -l) -eq 1 && + test $(git log --oneline Main_Page.mw | wc -l ) -eq 1 ) && wiki_check_content mw_dir_11/Nyan.mw Nyan && wiki_check_content mw_dir_11/Foo.mw Foo && @@ -218,9 +218,9 @@ test_expect_success 'Git clone works with the shallow option with a delete page' test_path_is_file mw_dir_12/Main_Page.mw && ( cd mw_dir_12 && - test `git log --oneline Nyan.mw | wc -l` -eq 1 && - test `git log --oneline Bar.mw | wc -l` -eq 1 && - test `git log --oneline Main_Page.mw | wc -l ` -eq 1 + test $(git log --oneline Nyan.mw | wc -l) -eq 1 && + test $(git log --oneline Bar.mw | wc -l) -eq 1 && + test $(git log --oneline Main_Page.mw | wc -l ) -eq 1 ) && wiki_check_content mw_dir_12/Nyan.mw Nyan && wiki_check_content mw_dir_12/Bar.mw Bar && diff --git a/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh b/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh index 37021e200a..6b0dbdac4d 100755 --- a/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh +++ b/contrib/mw-to-git/t/t9362-mw-to-git-utf8.sh @@ -70,8 +70,8 @@ test_expect_success 'The shallow option works with accents' ' test_path_is_file mw_dir_4/Main_Page.mw && ( cd mw_dir_4 && - test `git log --oneline Néoà .mw | wc -l` -eq 1 && - test `git log --oneline Main_Page.mw | wc -l ` -eq 1 + test $(git log --oneline Néoà .mw | wc -l) -eq 1 && + test $(git log --oneline Main_Page.mw | wc -l ) -eq 1 ) && wiki_check_content mw_dir_4/Néoà .mw Néoà && wiki_check_content mw_dir_4/Main_Page.mw Main_Page diff --git a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh index 5a0373935f..3ff3a09567 100755 --- a/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh +++ b/contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh @@ -58,6 +58,25 @@ test_expect_success 'git clone works on previously created wiki with media files test_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt ' +test_expect_success 'git push can upload media (File:) files containing valid UTF-8' ' + wiki_reset && + git clone mediawiki::'"$WIKI_URL"' mw_dir && + ( + cd mw_dir && + "$PERL_PATH" -e "print STDOUT \"UTF-8 content: éèà éê€.\";" >Bar.txt && + git add Bar.txt && + git commit -m "add a text file with UTF-8 content" && + git push + ) +' + +test_expect_success 'git clone works on previously created wiki with media files containing valid UTF-8' ' + test_when_finished "rm -rf mw_dir mw_dir_clone" && + git clone -c remote.origin.mediaimport=true \ + mediawiki::'"$WIKI_URL"' mw_dir_clone && + test_cmp mw_dir_clone/Bar.txt mw_dir/Bar.txt +' + test_expect_success 'git push & pull work with locally renamed media files' ' wiki_reset && git clone mediawiki::'"$WIKI_URL"' mw_dir && diff --git a/contrib/mw-to-git/t/t9365-continuing-queries.sh b/contrib/mw-to-git/t/t9365-continuing-queries.sh new file mode 100755 index 0000000000..016454749f --- /dev/null +++ b/contrib/mw-to-git/t/t9365-continuing-queries.sh @@ -0,0 +1,23 @@ +#!/bin/sh + +test_description='Test the Git Mediawiki remote helper: queries w/ more than 500 results' + +. ./test-gitmw-lib.sh +. $TEST_DIRECTORY/test-lib.sh + +test_check_precond + +test_expect_success 'creating page w/ >500 revisions' ' + wiki_reset && + for i in $(test_seq 501) + do + echo "creating revision $i" && + wiki_editpage foo "revision $i<br/>" true + done +' + +test_expect_success 'cloning page w/ >500 revisions' ' + git clone mediawiki::'"$WIKI_URL"' mw_dir +' + +test_done diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh index bb76cee379..6546294f15 100755 --- a/contrib/mw-to-git/t/test-gitmw-lib.sh +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh @@ -62,12 +62,8 @@ test_check_precond () { test_done fi - if [ ! -f "$GIT_BUILD_DIR"/git-remote-mediawiki ]; - then - echo "No remote mediawiki for git found. Copying it in git" - echo "cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/" - ln -s "$GIT_BUILD_DIR"/contrib/mw-to-git/git-remote-mediawiki "$GIT_BUILD_DIR" - fi + GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd "../.." && pwd) + PATH="$GIT_EXEC_PATH"'/bin-wrapper:'"$PATH" if [ ! -d "$WIKI_DIR_INST/$WIKI_DIR_NAME" ]; then @@ -94,8 +90,8 @@ test_diff_directories () { # # Check that <dir> contains exactly <N> files test_contains_N_files () { - if test `ls -- "$1" | wc -l` -ne "$2"; then - echo "directory $1 sould contain $2 files" + if test $(ls -- "$1" | wc -l) -ne "$2"; then + echo "directory $1 should contain $2 files" echo "it contains these files:" ls "$1" false @@ -293,7 +289,6 @@ start_lighttpd () { # Kill daemon lighttpd and removes files and folders associated. stop_lighttpd () { test -f "$WEB_TMP/pid" && kill $(cat "$WEB_TMP/pid") - rm -rf "$WEB" } # Create the SQLite database of the MediaWiki. If the database file already @@ -345,10 +340,10 @@ wiki_install () { "http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/"\ "$MW_FILENAME. "\ "Please fix your connection and launch the script again." - echo "$MW_FILENAME downloaded in `pwd`. "\ + echo "$MW_FILENAME downloaded in $(pwd). "\ "You can delete it later if you want." else - echo "Reusing existing $MW_FILENAME downloaded in `pwd`." + echo "Reusing existing $MW_FILENAME downloaded in $(pwd)." fi archive_abs_path=$(pwd)/$MW_FILENAME cd "$WIKI_DIR_INST/$WIKI_DIR_NAME/" || @@ -419,6 +414,7 @@ wiki_reset () { wiki_delete () { if test $LIGHTTPD = "true"; then stop_lighttpd + rm -fr "$WEB" else # Delete the wiki's directory. rm -rf "$WIKI_DIR_INST/$WIKI_DIR_NAME" || diff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config index 4cfebe9c69..5ba0684162 100644 --- a/contrib/mw-to-git/t/test.config +++ b/contrib/mw-to-git/t/test.config @@ -12,7 +12,7 @@ SERVER_ADDR=localhost TMP=/tmp DB_FILE=wikidb.sqlite -# If LIGHTTPD is not set to true, the script will use the defaut +# If LIGHTTPD is not set to true, the script will use the default # web server running in WIKI_DIR_INST. WIKI_DIR_INST=/var/www diff --git a/contrib/p4import/README b/contrib/p4import/README deleted file mode 100644 index b9892b6793..0000000000 --- a/contrib/p4import/README +++ /dev/null @@ -1 +0,0 @@ -Please see contrib/fast-import/git-p4 for a better Perforce importer. diff --git a/contrib/p4import/git-p4import.py b/contrib/p4import/git-p4import.py deleted file mode 100644 index 593d6a0682..0000000000 --- a/contrib/p4import/git-p4import.py +++ /dev/null @@ -1,365 +0,0 @@ -#!/usr/bin/env python -# -# This tool is copyright (c) 2006, Sean Estabrooks. -# It is released under the Gnu Public License, version 2. -# -# Import Perforce branches into Git repositories. -# Checking out the files is done by calling the standard p4 -# client which you must have properly configured yourself -# - -import marshal -import os -import sys -import time -import getopt - -if sys.hexversion < 0x02020000: - # The behavior of the marshal module changed significantly in 2.2 - sys.stderr.write("git-p4import.py: requires Python 2.2 or later.\n") - sys.exit(1) - -from signal import signal, \ - SIGPIPE, SIGINT, SIG_DFL, \ - default_int_handler - -signal(SIGPIPE, SIG_DFL) -s = signal(SIGINT, SIG_DFL) -if s != default_int_handler: - signal(SIGINT, s) - -def die(msg, *args): - for a in args: - msg = "%s %s" % (msg, a) - print "git-p4import fatal error:", msg - sys.exit(1) - -def usage(): - print "USAGE: git-p4import [-q|-v] [--authors=<file>] [-t <timezone>] [//p4repo/path <branch>]" - sys.exit(1) - -verbosity = 1 -logfile = "/dev/null" -ignore_warnings = False -stitch = 0 -tagall = True - -def report(level, msg, *args): - global verbosity - global logfile - for a in args: - msg = "%s %s" % (msg, a) - fd = open(logfile, "a") - fd.writelines(msg) - fd.close() - if level <= verbosity: - print msg - -class p4_command: - def __init__(self, _repopath): - try: - global logfile - self.userlist = {} - if _repopath[-1] == '/': - self.repopath = _repopath[:-1] - else: - self.repopath = _repopath - if self.repopath[-4:] != "/...": - self.repopath= "%s/..." % self.repopath - f=os.popen('p4 -V 2>>%s'%logfile, 'rb') - a = f.readlines() - if f.close(): - raise - except: - die("Could not find the \"p4\" command") - - def p4(self, cmd, *args): - global logfile - cmd = "%s %s" % (cmd, ' '.join(args)) - report(2, "P4:", cmd) - f=os.popen('p4 -G %s 2>>%s' % (cmd,logfile), 'rb') - list = [] - while 1: - try: - list.append(marshal.load(f)) - except EOFError: - break - self.ret = f.close() - return list - - def sync(self, id, force=False, trick=False, test=False): - if force: - ret = self.p4("sync -f %s@%s"%(self.repopath, id))[0] - elif trick: - ret = self.p4("sync -k %s@%s"%(self.repopath, id))[0] - elif test: - ret = self.p4("sync -n %s@%s"%(self.repopath, id))[0] - else: - ret = self.p4("sync %s@%s"%(self.repopath, id))[0] - if ret['code'] == "error": - data = ret['data'].upper() - if data.find('VIEW') > 0: - die("Perforce reports %s is not in client view"% self.repopath) - elif data.find('UP-TO-DATE') < 0: - die("Could not sync files from perforce", self.repopath) - - def changes(self, since=0): - try: - list = [] - for rec in self.p4("changes %s@%s,#head" % (self.repopath, since+1)): - list.append(rec['change']) - list.reverse() - return list - except: - return [] - - def authors(self, filename): - f=open(filename) - for l in f.readlines(): - self.userlist[l[:l.find('=')].rstrip()] = \ - (l[l.find('=')+1:l.find('<')].rstrip(),l[l.find('<')+1:l.find('>')]) - f.close() - for f,e in self.userlist.items(): - report(2, f, ":", e[0], " <", e[1], ">") - - def _get_user(self, id): - if not self.userlist.has_key(id): - try: - user = self.p4("users", id)[0] - self.userlist[id] = (user['FullName'], user['Email']) - except: - self.userlist[id] = (id, "") - return self.userlist[id] - - def _format_date(self, ticks): - symbol='+' - name = time.tzname[0] - offset = time.timezone - if ticks[8]: - name = time.tzname[1] - offset = time.altzone - if offset < 0: - offset *= -1 - symbol = '-' - localo = "%s%02d%02d %s" % (symbol, offset / 3600, offset % 3600, name) - tickso = time.strftime("%a %b %d %H:%M:%S %Y", ticks) - return "%s %s" % (tickso, localo) - - def where(self): - try: - return self.p4("where %s" % self.repopath)[-1]['path'] - except: - return "" - - def describe(self, num): - desc = self.p4("describe -s", num)[0] - self.msg = desc['desc'] - self.author, self.email = self._get_user(desc['user']) - self.date = self._format_date(time.localtime(long(desc['time']))) - return self - -class git_command: - def __init__(self): - try: - self.version = self.git("--version")[0][12:].rstrip() - except: - die("Could not find the \"git\" command") - try: - self.gitdir = self.get_single("rev-parse --git-dir") - report(2, "gdir:", self.gitdir) - except: - die("Not a git repository... did you forget to \"git init\" ?") - try: - self.cdup = self.get_single("rev-parse --show-cdup") - if self.cdup != "": - os.chdir(self.cdup) - self.topdir = os.getcwd() - report(2, "topdir:", self.topdir) - except: - die("Could not find top git directory") - - def git(self, cmd): - global logfile - report(2, "GIT:", cmd) - f=os.popen('git %s 2>>%s' % (cmd,logfile), 'rb') - r=f.readlines() - self.ret = f.close() - return r - - def get_single(self, cmd): - return self.git(cmd)[0].rstrip() - - def current_branch(self): - try: - testit = self.git("rev-parse --verify HEAD")[0] - return self.git("symbolic-ref HEAD")[0][11:].rstrip() - except: - return None - - def get_config(self, variable): - try: - return self.git("config --get %s" % variable)[0].rstrip() - except: - return None - - def set_config(self, variable, value): - try: - self.git("config %s %s"%(variable, value) ) - except: - die("Could not set %s to " % variable, value) - - def make_tag(self, name, head): - self.git("tag -f %s %s"%(name,head)) - - def top_change(self, branch): - try: - a=self.get_single("name-rev --tags refs/heads/%s" % branch) - loc = a.find(' tags/') + 6 - if a[loc:loc+3] != "p4/": - raise - return int(a[loc+3:][:-2]) - except: - return 0 - - def update_index(self): - self.git("ls-files -m -d -o -z | git update-index --add --remove -z --stdin") - - def checkout(self, branch): - self.git("checkout %s" % branch) - - def repoint_head(self, branch): - self.git("symbolic-ref HEAD refs/heads/%s" % branch) - - def remove_files(self): - self.git("ls-files | xargs rm") - - def clean_directories(self): - self.git("clean -d") - - def fresh_branch(self, branch): - report(1, "Creating new branch", branch) - self.git("ls-files | xargs rm") - os.remove(".git/index") - self.repoint_head(branch) - self.git("clean -d") - - def basedir(self): - return self.topdir - - def commit(self, author, email, date, msg, id): - self.update_index() - fd=open(".msg", "w") - fd.writelines(msg) - fd.close() - try: - current = self.get_single("rev-parse --verify HEAD") - head = "-p HEAD" - except: - current = "" - head = "" - tree = self.get_single("write-tree") - for r,l in [('DATE',date),('NAME',author),('EMAIL',email)]: - os.environ['GIT_AUTHOR_%s'%r] = l - os.environ['GIT_COMMITTER_%s'%r] = l - commit = self.get_single("commit-tree %s %s < .msg" % (tree,head)) - os.remove(".msg") - self.make_tag("p4/%s"%id, commit) - self.git("update-ref HEAD %s %s" % (commit, current) ) - -try: - opts, args = getopt.getopt(sys.argv[1:], "qhvt:", - ["authors=","help","stitch=","timezone=","log=","ignore","notags"]) -except getopt.GetoptError: - usage() - -for o, a in opts: - if o == "-q": - verbosity = 0 - if o == "-v": - verbosity += 1 - if o in ("--log"): - logfile = a - if o in ("--notags"): - tagall = False - if o in ("-h", "--help"): - usage() - if o in ("--ignore"): - ignore_warnings = True - -git = git_command() -branch=git.current_branch() - -for o, a in opts: - if o in ("-t", "--timezone"): - git.set_config("perforce.timezone", a) - if o in ("--stitch"): - git.set_config("perforce.%s.path" % branch, a) - stitch = 1 - -if len(args) == 2: - branch = args[1] - git.checkout(branch) - if branch == git.current_branch(): - die("Branch %s already exists!" % branch) - report(1, "Setting perforce to ", args[0]) - git.set_config("perforce.%s.path" % branch, args[0]) -elif len(args) != 0: - die("You must specify the perforce //depot/path and git branch") - -p4path = git.get_config("perforce.%s.path" % branch) -if p4path == None: - die("Do not know Perforce //depot/path for git branch", branch) - -p4 = p4_command(p4path) - -for o, a in opts: - if o in ("-a", "--authors"): - p4.authors(a) - -localdir = git.basedir() -if p4.where()[:len(localdir)] != localdir: - report(1, "**WARNING** Appears p4 client is misconfigured") - report(1, " for sync from %s to %s" % (p4.repopath, localdir)) - if ignore_warnings != True: - die("Reconfigure or use \"--ignore\" on command line") - -if stitch == 0: - top = git.top_change(branch) -else: - top = 0 -changes = p4.changes(top) -count = len(changes) -if count == 0: - report(1, "Already up to date...") - sys.exit(0) - -ptz = git.get_config("perforce.timezone") -if ptz: - report(1, "Setting timezone to", ptz) - os.environ['TZ'] = ptz - time.tzset() - -if stitch == 1: - git.remove_files() - git.clean_directories() - p4.sync(changes[0], force=True) -elif top == 0 and branch != git.current_branch(): - p4.sync(changes[0], test=True) - report(1, "Creating new initial commit"); - git.fresh_branch(branch) - p4.sync(changes[0], force=True) -else: - p4.sync(changes[0], trick=True) - -report(1, "processing %s changes from p4 (%s) to git (%s)" % (count, p4.repopath, branch)) -for id in changes: - report(1, "Importing changeset", id) - change = p4.describe(id) - p4.sync(id) - if tagall : - git.commit(change.author, change.email, change.date, change.msg, id) - else: - git.commit(change.author, change.email, change.date, change.msg, "import") - if stitch == 1: - git.clean_directories() - stitch = 0 diff --git a/contrib/p4import/git-p4import.txt b/contrib/p4import/git-p4import.txt deleted file mode 100644 index 9967587fe6..0000000000 --- a/contrib/p4import/git-p4import.txt +++ /dev/null @@ -1,167 +0,0 @@ -git-p4import(1) -=============== - -NAME ----- -git-p4import - Import a Perforce repository into git - - -SYNOPSIS --------- -[verse] -`git-p4import` [-q|-v] [--notags] [--authors <file>] [-t <timezone>] - <//p4repo/path> <branch> -`git-p4import` --stitch <//p4repo/path> -`git-p4import` - - -DESCRIPTION ------------ -Import a Perforce repository into an existing git repository. When -a <//p4repo/path> and <branch> are specified a new branch with the -given name will be created and the initial import will begin. - -Once the initial import is complete you can do an incremental import -of new commits from the Perforce repository. You do this by checking -out the appropriate git branch and then running `git-p4import` without -any options. - -The standard p4 client is used to communicate with the Perforce -repository; it must be configured correctly in order for `git-p4import` -to operate (see below). - - -OPTIONS -------- --q:: - Do not display any progress information. - --v:: - Give extra progress information. - -\--authors:: - Specify an authors file containing a mapping of Perforce user - ids to full names and email addresses (see Notes below). - -\--notags:: - Do not create a tag for each imported commit. - -\--stitch:: - Import the contents of the given perforce branch into the - currently checked out git branch. - -\--log:: - Store debugging information in the specified file. - --t:: - Specify that the remote repository is in the specified timezone. - Timezone must be in the format "US/Pacific" or "Europe/London" - etc. You only need to specify this once, it will be saved in - the git config file for the repository. - -<//p4repo/path>:: - The Perforce path that will be imported into the specified branch. - -<branch>:: - The new branch that will be created to hold the Perforce imports. - - -P4 Client ---------- -You must make the `p4` client command available in your $PATH and -configure it to communicate with the target Perforce repository. -Typically this means you must set the "$P4PORT" and "$P4CLIENT" -environment variables. - -You must also configure a `p4` client "view" which maps the Perforce -branch into the top level of your git repository, for example: - ------------- -Client: myhost - -Root: /home/sean/import - -Options: noallwrite clobber nocompress unlocked modtime rmdir - -View: - //public/jam/... //myhost/jam/... ------------- - -With the above `p4` client setup, you could import the "jam" -perforce branch into a branch named "jammy", like so: - ------------- -$ mkdir -p /home/sean/import/jam -$ cd /home/sean/import/jam -$ git init -$ git p4import //public/jam jammy ------------- - - -Multiple Branches ------------------ -Note that by creating multiple "views" you can use `git-p4import` -to import additional branches into the same git repository. -However, the `p4` client has a limitation in that it silently -ignores all but the last "view" that maps into the same local -directory. So the following will *not* work: - ------------- -View: - //public/jam/... //myhost/jam/... - //public/other/... //myhost/jam/... - //public/guest/... //myhost/jam/... ------------- - -If you want more than one Perforce branch to be imported into the -same directory you must employ a workaround. A simple option is -to adjust your `p4` client before each import to only include a -single view. - -Another option is to create multiple symlinks locally which all -point to the same directory in your git repository and then use -one per "view" instead of listing the actual directory. - - -Tags ----- -A git tag of the form p4/xx is created for every change imported from -the Perforce repository where xx is the Perforce changeset number. -Therefore after the import you can use git to access any commit by its -Perforce number, e.g. git show p4/327. - -The tag associated with the HEAD commit is also how `git-p4import` -determines if there are new changes to incrementally import from the -Perforce repository. - -If you import from a repository with many thousands of changes -you will have an equal number of p4/xxxx git tags. Git tags can -be expensive in terms of disk space and repository operations. -If you don't need to perform further incremental imports, you -may delete the tags. - - -Notes ------ -You can interrupt the import (e.g. ctrl-c) at any time and restart it -without worry. - -Author information is automatically determined by querying the -Perforce "users" table using the id associated with each change. -However, if you want to manually supply these mappings you can do -so with the "--authors" option. It accepts a file containing a list -of mappings with each line containing one mapping in the format: - ------------- - perforce_id = Full Name <email@address.com> ------------- - - -Author ------- -Written by Sean Estabrooks <seanlkml@sympatico.ca> - - -GIT ---- -Part of the gitlink:git[7] suite diff --git a/contrib/remote-helpers/Makefile b/contrib/remote-helpers/Makefile deleted file mode 100644 index 239161de33..0000000000 --- a/contrib/remote-helpers/Makefile +++ /dev/null @@ -1,14 +0,0 @@ -TESTS := $(wildcard test*.sh) - -export T := $(addprefix $(CURDIR)/,$(TESTS)) -export MAKE := $(MAKE) -e -export PATH := $(CURDIR):$(PATH) -export TEST_LINT := test-lint-executable test-lint-shell-syntax - -test: - $(MAKE) -C ../../t $@ - -$(TESTS): - $(MAKE) -C ../../t $(CURDIR)/$@ - -.PHONY: $(TESTS) diff --git a/contrib/remote-helpers/README b/contrib/remote-helpers/README new file mode 100644 index 0000000000..ac72332517 --- /dev/null +++ b/contrib/remote-helpers/README @@ -0,0 +1,15 @@ +The remote-helper bridges to access data stored in Mercurial and +Bazaar are maintained outside the git.git tree in the repositories +of their primary author: + + https://github.com/felipec/git-remote-hg (for Mercurial) + https://github.com/felipec/git-remote-bzr (for Bazaar) + +You can pick a directory on your $PATH and download them from these +repositories, e.g.: + + $ wget -O $HOME/bin/git-remote-hg \ + https://raw.github.com/felipec/git-remote-hg/master/git-remote-hg + $ wget -O $HOME/bin/git-remote-bzr \ + https://raw.github.com/felipec/git-remote-bzr/master/git-remote-bzr + $ chmod +x $HOME/bin/git-remote-hg $HOME/bin/git-remote-bzr diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr index c3a3cac77b..712a1377e2 100755 --- a/contrib/remote-helpers/git-remote-bzr +++ b/contrib/remote-helpers/git-remote-bzr @@ -1,960 +1,13 @@ #!/usr/bin/env python -# -# Copyright (c) 2012 Felipe Contreras -# - -# -# Just copy to your ~/bin, or anywhere in your $PATH. -# Then you can clone with: -# % git clone bzr::/path/to/bzr/repo/or/url -# -# For example: -# % git clone bzr::$HOME/myrepo -# or -# % git clone bzr::lp:myrepo -# -# If you want to specify which branches you want track (per repo): -# git config remote-bzr.branches 'trunk, devel, test' -# - -import sys - -import bzrlib -if hasattr(bzrlib, "initialize"): - bzrlib.initialize() - -import bzrlib.plugin -bzrlib.plugin.load_plugins() - -import bzrlib.generate_ids -import bzrlib.transport -import bzrlib.errors -import bzrlib.ui -import bzrlib.urlutils -import bzrlib.branch import sys -import os -import json -import re -import StringIO -import atexit, shutil, hashlib, urlparse, subprocess - -NAME_RE = re.compile('^([^<>]+)') -AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]*)>$') -EMAIL_RE = re.compile('^([^<>]+[^ \\\t<>])?\\b(?:[ \\t<>]*?)\\b([^ \\t<>]+@[^ \\t<>]+)') -RAW_AUTHOR_RE = re.compile('^(\w+) (.+)? <(.*)> (\d+) ([+-]\d+)') - -def die(msg, *args): - sys.stderr.write('ERROR: %s\n' % (msg % args)) - sys.exit(1) - -def warn(msg, *args): - sys.stderr.write('WARNING: %s\n' % (msg % args)) - -def gittz(tz): - return '%+03d%02d' % (tz / 3600, tz % 3600 / 60) - -def get_config(config): - cmd = ['git', 'config', '--get', config] - process = subprocess.Popen(cmd, stdout=subprocess.PIPE) - output, _ = process.communicate() - return output - -class Marks: - - def __init__(self, path): - self.path = path - self.tips = {} - self.marks = {} - self.rev_marks = {} - self.last_mark = 0 - self.load() - - def load(self): - if not os.path.exists(self.path): - return - - tmp = json.load(open(self.path)) - self.tips = tmp['tips'] - self.marks = tmp['marks'] - self.last_mark = tmp['last-mark'] - - for rev, mark in self.marks.iteritems(): - self.rev_marks[mark] = rev - - def dict(self): - return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark } - - def store(self): - json.dump(self.dict(), open(self.path, 'w')) - - def __str__(self): - return str(self.dict()) - - def from_rev(self, rev): - return self.marks[rev] - - def to_rev(self, mark): - return str(self.rev_marks[mark]) - - def next_mark(self): - self.last_mark += 1 - return self.last_mark - - def get_mark(self, rev): - self.last_mark += 1 - self.marks[rev] = self.last_mark - return self.last_mark - - def is_marked(self, rev): - return rev in self.marks - - def new_mark(self, rev, mark): - self.marks[rev] = mark - self.rev_marks[mark] = rev - self.last_mark = mark - - def get_tip(self, branch): - try: - return str(self.tips[branch]) - except KeyError: - return None - - def set_tip(self, branch, tip): - self.tips[branch] = tip - -class Parser: - - def __init__(self, repo): - self.repo = repo - self.line = self.get_line() - - def get_line(self): - return sys.stdin.readline().strip() - - def __getitem__(self, i): - return self.line.split()[i] - - def check(self, word): - return self.line.startswith(word) - - def each_block(self, separator): - while self.line != separator: - yield self.line - self.line = self.get_line() - - def __iter__(self): - return self.each_block('') - - def next(self): - self.line = self.get_line() - if self.line == 'done': - self.line = None - - def get_mark(self): - i = self.line.index(':') + 1 - return int(self.line[i:]) - - def get_data(self): - if not self.check('data'): - return None - i = self.line.index(' ') + 1 - size = int(self.line[i:]) - return sys.stdin.read(size) - - def get_author(self): - m = RAW_AUTHOR_RE.match(self.line) - if not m: - return None - _, name, email, date, tz = m.groups() - committer = '%s <%s>' % (name, email) - tz = int(tz) - tz = ((tz / 100) * 3600) + ((tz % 100) * 60) - return (committer, int(date), tz) - -def rev_to_mark(rev): - global marks - return marks.from_rev(rev) - -def mark_to_rev(mark): - global marks - return marks.to_rev(mark) - -def fixup_user(user): - name = mail = None - user = user.replace('"', '') - m = AUTHOR_RE.match(user) - if m: - name = m.group(1) - mail = m.group(2).strip() - else: - m = EMAIL_RE.match(user) - if m: - name = m.group(1) - mail = m.group(2) - else: - m = NAME_RE.match(user) - if m: - name = m.group(1).strip() - - if not name: - name = 'unknown' - if not mail: - mail = 'Unknown' - - return '%s <%s>' % (name, mail) - -def get_filechanges(cur, prev): - modified = {} - removed = {} - - changes = cur.changes_from(prev) - - def u(s): - return s.encode('utf-8') - - for path, fid, kind in changes.added: - modified[u(path)] = fid - for path, fid, kind in changes.removed: - removed[u(path)] = None - for path, fid, kind, mod, _ in changes.modified: - modified[u(path)] = fid - for oldpath, newpath, fid, kind, mod, _ in changes.renamed: - removed[u(oldpath)] = None - if kind == 'directory': - lst = cur.list_files(from_dir=newpath, recursive=True) - for path, file_class, kind, fid, entry in lst: - if kind != 'directory': - modified[u(newpath + '/' + path)] = fid - else: - modified[u(newpath)] = fid - - return modified, removed - -def export_files(tree, files): - global marks, filenodes - - final = [] - for path, fid in files.iteritems(): - kind = tree.kind(fid) - - h = tree.get_file_sha1(fid) - - if kind == 'symlink': - d = tree.get_symlink_target(fid) - mode = '120000' - elif kind == 'file': - - if tree.is_executable(fid): - mode = '100755' - else: - mode = '100644' - - # is the blob already exported? - if h in filenodes: - mark = filenodes[h] - final.append((mode, mark, path)) - continue - - d = tree.get_file_text(fid) - elif kind == 'directory': - continue - else: - die("Unhandled kind '%s' for path '%s'" % (kind, path)) - - mark = marks.next_mark() - filenodes[h] = mark - - print "blob" - print "mark :%u" % mark - print "data %d" % len(d) - print d - - final.append((mode, mark, path)) - - return final - -def export_branch(repo, name): - global prefix - - ref = '%s/heads/%s' % (prefix, name) - tip = marks.get_tip(name) - - branch = get_remote_branch(name) - repo = branch.repository - - branch.lock_read() - revs = branch.iter_merge_sorted_revisions(None, tip, 'exclude', 'forward') - try: - tip_revno = branch.revision_id_to_revno(tip) - last_revno, _ = branch.last_revision_info() - total = last_revno - tip_revno - except bzrlib.errors.NoSuchRevision: - tip_revno = 0 - total = 0 - - for revid, _, seq, _ in revs: - - if marks.is_marked(revid): - continue - - rev = repo.get_revision(revid) - revno = seq[0] - - parents = rev.parent_ids - time = rev.timestamp - tz = rev.timezone - committer = rev.committer.encode('utf-8') - committer = "%s %u %s" % (fixup_user(committer), time, gittz(tz)) - authors = rev.get_apparent_authors() - if authors: - author = authors[0].encode('utf-8') - author = "%s %u %s" % (fixup_user(author), time, gittz(tz)) - else: - author = committer - msg = rev.message.encode('utf-8') - - msg += '\n' - - if len(parents) == 0: - parent = bzrlib.revision.NULL_REVISION - else: - parent = parents[0] - - cur_tree = repo.revision_tree(revid) - prev = repo.revision_tree(parent) - modified, removed = get_filechanges(cur_tree, prev) - - modified_final = export_files(cur_tree, modified) - - if len(parents) == 0: - print 'reset %s' % ref - - print "commit %s" % ref - print "mark :%d" % (marks.get_mark(revid)) - print "author %s" % (author) - print "committer %s" % (committer) - print "data %d" % (len(msg)) - print msg - - for i, p in enumerate(parents): - try: - m = rev_to_mark(p) - except KeyError: - # ghost? - continue - if i == 0: - print "from :%s" % m - else: - print "merge :%s" % m - - for f in removed: - print "D %s" % (f,) - for f in modified_final: - print "M %s :%u %s" % f - print - - if len(seq) > 1: - # let's skip branch revisions from the progress report - continue - - progress = (revno - tip_revno) - if (progress % 100 == 0): - if total: - print "progress revision %d '%s' (%d/%d)" % (revno, name, progress, total) - else: - print "progress revision %d '%s' (%d)" % (revno, name, progress) - - branch.unlock() - - revid = branch.last_revision() - - # make sure the ref is updated - print "reset %s" % ref - print "from :%u" % rev_to_mark(revid) - print - - marks.set_tip(name, revid) - -def export_tag(repo, name): - global tags, prefix - - ref = '%s/tags/%s' % (prefix, name) - print "reset %s" % ref - print "from :%u" % rev_to_mark(tags[name]) - print - -def do_import(parser): - global dirname - - repo = parser.repo - path = os.path.join(dirname, 'marks-git') - - print "feature done" - if os.path.exists(path): - print "feature import-marks=%s" % path - print "feature export-marks=%s" % path - print "feature force" - sys.stdout.flush() - - while parser.check('import'): - ref = parser[1] - if ref.startswith('refs/heads/'): - name = ref[len('refs/heads/'):] - export_branch(repo, name) - if ref.startswith('refs/tags/'): - name = ref[len('refs/tags/'):] - export_tag(repo, name) - parser.next() - - print 'done' - - sys.stdout.flush() - -def parse_blob(parser): - global blob_marks - - parser.next() - mark = parser.get_mark() - parser.next() - data = parser.get_data() - blob_marks[mark] = data - parser.next() - -class CustomTree(): - - def __init__(self, branch, revid, parents, files): - global files_cache - - self.updates = {} - self.branch = branch - - def copy_tree(revid): - files = files_cache[revid] = {} - branch.lock_read() - tree = branch.repository.revision_tree(revid) - try: - for path, entry in tree.iter_entries_by_dir(): - files[path] = [entry.file_id, None] - finally: - branch.unlock() - return files - - if len(parents) == 0: - self.base_id = bzrlib.revision.NULL_REVISION - self.base_files = {} - else: - self.base_id = parents[0] - self.base_files = files_cache.get(self.base_id, None) - if not self.base_files: - self.base_files = copy_tree(self.base_id) - - self.files = files_cache[revid] = self.base_files.copy() - self.rev_files = {} - - for path, data in self.files.iteritems(): - fid, mark = data - self.rev_files[fid] = [path, mark] - - for path, f in files.iteritems(): - fid, mark = self.files.get(path, [None, None]) - if not fid: - fid = bzrlib.generate_ids.gen_file_id(path) - f['path'] = path - self.rev_files[fid] = [path, mark] - self.updates[fid] = f - - def last_revision(self): - return self.base_id - - def iter_changes(self): - changes = [] - - def get_parent(dirname, basename): - parent_fid, mark = self.base_files.get(dirname, [None, None]) - if parent_fid: - return parent_fid - parent_fid, mark = self.files.get(dirname, [None, None]) - if parent_fid: - return parent_fid - if basename == '': - return None - fid = bzrlib.generate_ids.gen_file_id(path) - add_entry(fid, dirname, 'directory') - return fid - - def add_entry(fid, path, kind, mode = None): - dirname, basename = os.path.split(path) - parent_fid = get_parent(dirname, basename) - - executable = False - if mode == '100755': - executable = True - elif mode == '120000': - kind = 'symlink' - - change = (fid, - (None, path), - True, - (False, True), - (None, parent_fid), - (None, basename), - (None, kind), - (None, executable)) - self.files[path] = [change[0], None] - changes.append(change) - - def update_entry(fid, path, kind, mode = None): - dirname, basename = os.path.split(path) - parent_fid = get_parent(dirname, basename) - - executable = False - if mode == '100755': - executable = True - elif mode == '120000': - kind = 'symlink' - - change = (fid, - (path, path), - True, - (True, True), - (None, parent_fid), - (None, basename), - (None, kind), - (None, executable)) - self.files[path] = [change[0], None] - changes.append(change) - - def remove_entry(fid, path, kind): - dirname, basename = os.path.split(path) - parent_fid = get_parent(dirname, basename) - change = (fid, - (path, None), - True, - (True, False), - (parent_fid, None), - (None, None), - (None, None), - (None, None)) - del self.files[path] - changes.append(change) - - for fid, f in self.updates.iteritems(): - path = f['path'] - - if 'deleted' in f: - remove_entry(fid, path, 'file') - continue - - if path in self.base_files: - update_entry(fid, path, 'file', f['mode']) - else: - add_entry(fid, path, 'file', f['mode']) - - self.files[path][1] = f['mark'] - self.rev_files[fid][1] = f['mark'] - - return changes - - def get_content(self, file_id): - path, mark = self.rev_files[file_id] - if mark: - return blob_marks[mark] - - # last resort - tree = self.branch.repository.revision_tree(self.base_id) - return tree.get_file_text(file_id) - - def get_file_with_stat(self, file_id, path=None): - content = self.get_content(file_id) - return (StringIO.StringIO(content), None) - - def get_symlink_target(self, file_id): - return self.get_content(file_id) - - def id2path(self, file_id): - path, mark = self.rev_files[file_id] - return path - -def c_style_unescape(string): - if string[0] == string[-1] == '"': - return string.decode('string-escape')[1:-1] - return string - -def parse_commit(parser): - global marks, blob_marks, parsed_refs - global mode - - parents = [] - - ref = parser[1] - parser.next() - - if ref.startswith('refs/heads/'): - name = ref[len('refs/heads/'):] - branch = get_remote_branch(name) - else: - die('unknown ref') - - commit_mark = parser.get_mark() - parser.next() - author = parser.get_author() - parser.next() - committer = parser.get_author() - parser.next() - data = parser.get_data() - parser.next() - if parser.check('from'): - parents.append(parser.get_mark()) - parser.next() - while parser.check('merge'): - parents.append(parser.get_mark()) - parser.next() - - # fast-export adds an extra newline - if data[-1] == '\n': - data = data[:-1] - - files = {} - - for line in parser: - if parser.check('M'): - t, m, mark_ref, path = line.split(' ', 3) - mark = int(mark_ref[1:]) - f = { 'mode' : m, 'mark' : mark } - elif parser.check('D'): - t, path = line.split(' ', 1) - f = { 'deleted' : True } - else: - die('Unknown file command: %s' % line) - path = c_style_unescape(path).decode('utf-8') - files[path] = f - - committer, date, tz = committer - parents = [mark_to_rev(p) for p in parents] - revid = bzrlib.generate_ids.gen_revision_id(committer, date) - props = {} - props['branch-nick'] = branch.nick - - mtree = CustomTree(branch, revid, parents, files) - changes = mtree.iter_changes() - - branch.lock_write() - try: - builder = branch.get_commit_builder(parents, None, date, tz, committer, props, revid) - try: - list(builder.record_iter_changes(mtree, mtree.last_revision(), changes)) - builder.finish_inventory() - builder.commit(data.decode('utf-8', 'replace')) - except Exception, e: - builder.abort() - raise - finally: - branch.unlock() - - parsed_refs[ref] = revid - marks.new_mark(revid, commit_mark) - -def parse_reset(parser): - global parsed_refs - - ref = parser[1] - parser.next() - - # ugh - if parser.check('commit'): - parse_commit(parser) - return - if not parser.check('from'): - return - from_mark = parser.get_mark() - parser.next() - - parsed_refs[ref] = mark_to_rev(from_mark) - -def do_export(parser): - global parsed_refs, dirname - - parser.next() - - for line in parser.each_block('done'): - if parser.check('blob'): - parse_blob(parser) - elif parser.check('commit'): - parse_commit(parser) - elif parser.check('reset'): - parse_reset(parser) - elif parser.check('tag'): - pass - elif parser.check('feature'): - pass - else: - die('unhandled export command: %s' % line) - - for ref, revid in parsed_refs.iteritems(): - if ref.startswith('refs/heads/'): - name = ref[len('refs/heads/'):] - branch = get_remote_branch(name) - branch.generate_revision_history(revid, marks.get_tip(name)) - - if name in peers: - peer = bzrlib.branch.Branch.open(peers[name]) - try: - peer.bzrdir.push_branch(branch, revision_id=revid) - except bzrlib.errors.DivergedBranches: - print "error %s non-fast forward" % ref - continue - - try: - wt = branch.bzrdir.open_workingtree() - wt.update() - except bzrlib.errors.NoWorkingTree: - pass - elif ref.startswith('refs/tags/'): - # TODO: implement tag push - print "error %s pushing tags not supported" % ref - continue - else: - # transport-helper/fast-export bugs - continue - - print "ok %s" % ref - - print - -def do_capabilities(parser): - global dirname - - print "import" - print "export" - print "refspec refs/heads/*:%s/heads/*" % prefix - print "refspec refs/tags/*:%s/tags/*" % prefix - - path = os.path.join(dirname, 'marks-git') - - if os.path.exists(path): - print "*import-marks %s" % path - print "*export-marks %s" % path - - print - -def ref_is_valid(name): - return not True in [c in name for c in '~^: \\'] - -def do_list(parser): - global tags - - master_branch = None - - for name in branches: - if not master_branch: - master_branch = name - print "? refs/heads/%s" % name - - branch = get_remote_branch(master_branch) - branch.lock_read() - for tag, revid in branch.tags.get_tag_dict().items(): - try: - branch.revision_id_to_dotted_revno(revid) - except bzrlib.errors.NoSuchRevision: - continue - if not ref_is_valid(tag): - continue - print "? refs/tags/%s" % tag - tags[tag] = revid - branch.unlock() - - print "@refs/heads/%s HEAD" % master_branch - print - -def clone(path, remote_branch): - try: - bdir = bzrlib.bzrdir.BzrDir.create(path) - except bzrlib.errors.AlreadyControlDirError: - bdir = bzrlib.bzrdir.BzrDir.open(path) - repo = bdir.find_repository() - repo.fetch(remote_branch.repository) - return remote_branch.sprout(bdir, repository=repo) - -def get_remote_branch(name): - global dirname, branches - - remote_branch = bzrlib.branch.Branch.open(branches[name]) - if isinstance(remote_branch.user_transport, bzrlib.transport.local.LocalTransport): - return remote_branch - - branch_path = os.path.join(dirname, 'clone', name) - - try: - branch = bzrlib.branch.Branch.open(branch_path) - except bzrlib.errors.NotBranchError: - # clone - branch = clone(branch_path, remote_branch) - else: - # pull - try: - branch.pull(remote_branch, overwrite=True) - except bzrlib.errors.DivergedBranches: - # use remote branch for now - return remote_branch - - return branch - -def find_branches(repo): - transport = repo.bzrdir.root_transport - - for fn in transport.iter_files_recursive(): - if not fn.endswith('.bzr/branch-format'): - continue - - name = subdir = fn[:-len('/.bzr/branch-format')] - name = name if name != '' else 'master' - name = name.replace('/', '+') - - try: - cur = transport.clone(subdir) - branch = bzrlib.branch.Branch.open_from_transport(cur) - except bzrlib.errors.NotBranchError: - continue - else: - yield name, branch.base - -def get_repo(url, alias): - global dirname, peer, branches - - normal_url = bzrlib.urlutils.normalize_url(url) - origin = bzrlib.bzrdir.BzrDir.open(url) - is_local = isinstance(origin.transport, bzrlib.transport.local.LocalTransport) - - shared_path = os.path.join(gitdir, 'bzr') - try: - shared_dir = bzrlib.bzrdir.BzrDir.open(shared_path) - except bzrlib.errors.NotBranchError: - shared_dir = bzrlib.bzrdir.BzrDir.create(shared_path) - try: - shared_repo = shared_dir.open_repository() - except bzrlib.errors.NoRepositoryPresent: - shared_repo = shared_dir.create_repository(shared=True) - - if not is_local: - clone_path = os.path.join(dirname, 'clone') - if not os.path.exists(clone_path): - os.mkdir(clone_path) - else: - # check and remove old organization - try: - bdir = bzrlib.bzrdir.BzrDir.open(clone_path) - bdir.destroy_repository() - except bzrlib.errors.NotBranchError: - pass - except bzrlib.errors.NoRepositoryPresent: - pass - - wanted = get_config('remote-bzr.branches').rstrip().split(', ') - # stupid python - wanted = [e for e in wanted if e] - - if not wanted: - try: - repo = origin.open_repository() - if not repo.user_transport.listable(): - # this repository is not usable for us - raise bzrlib.errors.NoRepositoryPresent(repo.bzrdir) - except bzrlib.errors.NoRepositoryPresent: - wanted = ['master'] - - if wanted: - def list_wanted(url, wanted): - for name in wanted: - subdir = name if name != 'master' else '' - yield name, bzrlib.urlutils.join(url, subdir) - - branch_list = list_wanted(url, wanted) - else: - branch_list = find_branches(repo) - - for name, url in branch_list: - if not is_local: - peers[name] = url - branches[name] = url - - return origin - -def fix_path(alias, orig_url): - url = urlparse.urlparse(orig_url, 'file') - if url.scheme != 'file' or os.path.isabs(url.path): - return - abs_url = urlparse.urljoin("%s/" % os.getcwd(), orig_url) - cmd = ['git', 'config', 'remote.%s.url' % alias, "bzr::%s" % abs_url] - subprocess.call(cmd) - -def main(args): - global marks, prefix, gitdir, dirname - global tags, filenodes - global blob_marks - global parsed_refs - global files_cache - global is_tmp - global branches, peers - - alias = args[1] - url = args[2] - - tags = {} - filenodes = {} - blob_marks = {} - parsed_refs = {} - files_cache = {} - marks = None - branches = {} - peers = {} - - if alias[5:] == url: - is_tmp = True - alias = hashlib.sha1(alias).hexdigest() - else: - is_tmp = False - - prefix = 'refs/bzr/%s' % alias - gitdir = os.environ['GIT_DIR'] - dirname = os.path.join(gitdir, 'bzr', alias) - - if not is_tmp: - fix_path(alias, url) - - if not os.path.exists(dirname): - os.makedirs(dirname) - - if hasattr(bzrlib.ui.ui_factory, 'be_quiet'): - bzrlib.ui.ui_factory.be_quiet(True) - - repo = get_repo(url, alias) - - marks_path = os.path.join(dirname, 'marks-int') - marks = Marks(marks_path) - - parser = Parser(repo) - for line in parser: - if parser.check('capabilities'): - do_capabilities(parser) - elif parser.check('list'): - do_list(parser) - elif parser.check('import'): - do_import(parser) - elif parser.check('export'): - do_export(parser) - else: - die('unhandled command: %s' % line) - sys.stdout.flush() -def bye(): - if not marks: - return - if not is_tmp: - marks.store() - else: - shutil.rmtree(dirname) +sys.stderr.write('WARNING: git-remote-bzr is now maintained independently.\n') +sys.stderr.write('WARNING: For more information visit https://github.com/felipec/git-remote-bzr\n') -atexit.register(bye) -sys.exit(main(sys.argv)) +sys.stderr.write('''WARNING: +WARNING: You can pick a directory on your $PATH and download it, e.g.: +WARNING: $ wget -O $HOME/bin/git-remote-bzr \\ +WARNING: https://raw.github.com/felipec/git-remote-bzr/master/git-remote-bzr +WARNING: $ chmod +x $HOME/bin/git-remote-bzr +''') diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg index 0194c67fb1..4255ad6312 100755 --- a/contrib/remote-helpers/git-remote-hg +++ b/contrib/remote-helpers/git-remote-hg @@ -1,1220 +1,13 @@ #!/usr/bin/env python -# -# Copyright (c) 2012 Felipe Contreras -# -# Inspired by Rocco Rutte's hg-fast-export - -# Just copy to your ~/bin, or anywhere in your $PATH. -# Then you can clone with: -# git clone hg::/path/to/mercurial/repo/ -# -# For remote repositories a local clone is stored in -# "$GIT_DIR/hg/origin/clone/.hg/". - -from mercurial import hg, ui, bookmarks, context, encoding, node, error, extensions, discovery, util - -import re import sys -import os -import json -import shutil -import subprocess -import urllib -import atexit -import urlparse, hashlib - -# -# If you are not in hg-git-compat mode and want to disable the tracking of -# named branches: -# git config --global remote-hg.track-branches false -# -# If you want the equivalent of hg's clone/pull--insecure option: -# git config --global remote-hg.insecure true -# -# If you want to switch to hg-git compatibility mode: -# git config --global remote-hg.hg-git-compat true -# -# git: -# Sensible defaults for git. -# hg bookmarks are exported as git branches, hg branches are prefixed -# with 'branches/', HEAD is a special case. -# -# hg: -# Emulate hg-git. -# Only hg bookmarks are exported as git branches. -# Commits are modified to preserve hg information and allow bidirectionality. -# - -NAME_RE = re.compile('^([^<>]+)') -AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]*)>$') -EMAIL_RE = re.compile('^([^<>]+[^ \\\t<>])?\\b(?:[ \\t<>]*?)\\b([^ \\t<>]+@[^ \\t<>]+)') -AUTHOR_HG_RE = re.compile('^(.*?) ?<(.*?)(?:>(.+)?)?$') -RAW_AUTHOR_RE = re.compile('^(\w+) (?:(.+)? )?<(.*)> (\d+) ([+-]\d+)') - -VERSION = 2 - -def die(msg, *args): - sys.stderr.write('ERROR: %s\n' % (msg % args)) - sys.exit(1) - -def warn(msg, *args): - sys.stderr.write('WARNING: %s\n' % (msg % args)) - -def gitmode(flags): - return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644' - -def gittz(tz): - return '%+03d%02d' % (-tz / 3600, -tz % 3600 / 60) - -def hgmode(mode): - m = { '100755': 'x', '120000': 'l' } - return m.get(mode, '') - -def hghex(n): - return node.hex(n) - -def hgbin(n): - return node.bin(n) - -def hgref(ref): - return ref.replace('___', ' ') - -def gitref(ref): - return ref.replace(' ', '___') - -def check_version(*check): - if not hg_version: - return True - return hg_version >= check - -def get_config(config): - cmd = ['git', 'config', '--get', config] - process = subprocess.Popen(cmd, stdout=subprocess.PIPE) - output, _ = process.communicate() - return output - -def get_config_bool(config, default=False): - value = get_config(config).rstrip('\n') - if value == "true": - return True - elif value == "false": - return False - else: - return default - -class Marks: - - def __init__(self, path, repo): - self.path = path - self.repo = repo - self.clear() - self.load() - - if self.version < VERSION: - if self.version == 1: - self.upgrade_one() - - # upgraded? - if self.version < VERSION: - self.clear() - self.version = VERSION - - def clear(self): - self.tips = {} - self.marks = {} - self.rev_marks = {} - self.last_mark = 0 - self.version = 0 - - def load(self): - if not os.path.exists(self.path): - return - - tmp = json.load(open(self.path)) - - self.tips = tmp['tips'] - self.marks = tmp['marks'] - self.last_mark = tmp['last-mark'] - self.version = tmp.get('version', 1) - - for rev, mark in self.marks.iteritems(): - self.rev_marks[mark] = rev - - def upgrade_one(self): - def get_id(rev): - return hghex(self.repo.changelog.node(int(rev))) - self.tips = dict((name, get_id(rev)) for name, rev in self.tips.iteritems()) - self.marks = dict((get_id(rev), mark) for rev, mark in self.marks.iteritems()) - self.rev_marks = dict((mark, get_id(rev)) for mark, rev in self.rev_marks.iteritems()) - self.version = 2 - - def dict(self): - return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark, 'version' : self.version } - - def store(self): - json.dump(self.dict(), open(self.path, 'w')) - - def __str__(self): - return str(self.dict()) - - def from_rev(self, rev): - return self.marks[rev] - - def to_rev(self, mark): - return str(self.rev_marks[mark]) - - def next_mark(self): - self.last_mark += 1 - return self.last_mark - - def get_mark(self, rev): - self.last_mark += 1 - self.marks[rev] = self.last_mark - return self.last_mark - - def new_mark(self, rev, mark): - self.marks[rev] = mark - self.rev_marks[mark] = rev - self.last_mark = mark - - def is_marked(self, rev): - return rev in self.marks - - def get_tip(self, branch): - return str(self.tips[branch]) - - def set_tip(self, branch, tip): - self.tips[branch] = tip - -class Parser: - - def __init__(self, repo): - self.repo = repo - self.line = self.get_line() - - def get_line(self): - return sys.stdin.readline().strip() - - def __getitem__(self, i): - return self.line.split()[i] - - def check(self, word): - return self.line.startswith(word) - - def each_block(self, separator): - while self.line != separator: - yield self.line - self.line = self.get_line() - - def __iter__(self): - return self.each_block('') - - def next(self): - self.line = self.get_line() - if self.line == 'done': - self.line = None - - def get_mark(self): - i = self.line.index(':') + 1 - return int(self.line[i:]) - - def get_data(self): - if not self.check('data'): - return None - i = self.line.index(' ') + 1 - size = int(self.line[i:]) - return sys.stdin.read(size) - - def get_author(self): - global bad_mail - - ex = None - m = RAW_AUTHOR_RE.match(self.line) - if not m: - return None - _, name, email, date, tz = m.groups() - if name and 'ext:' in name: - m = re.match('^(.+?) ext:\((.+)\)$', name) - if m: - name = m.group(1) - ex = urllib.unquote(m.group(2)) - - if email != bad_mail: - if name: - user = '%s <%s>' % (name, email) - else: - user = '<%s>' % (email) - else: - user = name - - if ex: - user += ex - - tz = int(tz) - tz = ((tz / 100) * 3600) + ((tz % 100) * 60) - return (user, int(date), -tz) - -def fix_file_path(path): - if not os.path.isabs(path): - return path - return os.path.relpath(path, '/') - -def export_files(files): - global marks, filenodes - - final = [] - for f in files: - fid = node.hex(f.filenode()) - - if fid in filenodes: - mark = filenodes[fid] - else: - mark = marks.next_mark() - filenodes[fid] = mark - d = f.data() - - print "blob" - print "mark :%u" % mark - print "data %d" % len(d) - print d - - path = fix_file_path(f.path()) - final.append((gitmode(f.flags()), mark, path)) - - return final - -def get_filechanges(repo, ctx, parent): - modified = set() - added = set() - removed = set() - - # load earliest manifest first for caching reasons - prev = parent.manifest().copy() - cur = ctx.manifest() - - for fn in cur: - if fn in prev: - if (cur.flags(fn) != prev.flags(fn) or cur[fn] != prev[fn]): - modified.add(fn) - del prev[fn] - else: - added.add(fn) - removed |= set(prev.keys()) - - return added | modified, removed - -def fixup_user_git(user): - name = mail = None - user = user.replace('"', '') - m = AUTHOR_RE.match(user) - if m: - name = m.group(1) - mail = m.group(2).strip() - else: - m = EMAIL_RE.match(user) - if m: - name = m.group(1) - mail = m.group(2) - else: - m = NAME_RE.match(user) - if m: - name = m.group(1).strip() - return (name, mail) - -def fixup_user_hg(user): - def sanitize(name): - # stole this from hg-git - return re.sub('[<>\n]', '?', name.lstrip('< ').rstrip('> ')) - - m = AUTHOR_HG_RE.match(user) - if m: - name = sanitize(m.group(1)) - mail = sanitize(m.group(2)) - ex = m.group(3) - if ex: - name += ' ext:(' + urllib.quote(ex) + ')' - else: - name = sanitize(user) - if '@' in user: - mail = name - else: - mail = None - - return (name, mail) - -def fixup_user(user): - global mode, bad_mail - - if mode == 'git': - name, mail = fixup_user_git(user) - else: - name, mail = fixup_user_hg(user) - - if not name: - name = bad_name - if not mail: - mail = bad_mail - - return '%s <%s>' % (name, mail) - -def updatebookmarks(repo, peer): - remotemarks = peer.listkeys('bookmarks') - localmarks = repo._bookmarks - - if not remotemarks: - return - - for k, v in remotemarks.iteritems(): - localmarks[k] = hgbin(v) - - if hasattr(localmarks, 'write'): - localmarks.write() - else: - bookmarks.write(repo) - -def get_repo(url, alias): - global dirname, peer - - myui = ui.ui() - myui.setconfig('ui', 'interactive', 'off') - myui.fout = sys.stderr - - if get_config_bool('remote-hg.insecure'): - myui.setconfig('web', 'cacerts', '') - - extensions.loadall(myui) - - if hg.islocal(url) and not os.environ.get('GIT_REMOTE_HG_TEST_REMOTE'): - repo = hg.repository(myui, url) - if not os.path.exists(dirname): - os.makedirs(dirname) - else: - shared_path = os.path.join(gitdir, 'hg') - if not os.path.exists(shared_path): - try: - hg.clone(myui, {}, url, shared_path, update=False, pull=True) - except: - die('Repository error') - - if not os.path.exists(dirname): - os.makedirs(dirname) - - local_path = os.path.join(dirname, 'clone') - if not os.path.exists(local_path): - hg.share(myui, shared_path, local_path, update=False) - - repo = hg.repository(myui, local_path) - try: - peer = hg.peer(myui, {}, url) - except: - die('Repository error') - repo.pull(peer, heads=None, force=True) - - updatebookmarks(repo, peer) - - return repo - -def rev_to_mark(rev): - global marks - return marks.from_rev(rev.hex()) - -def mark_to_rev(mark): - global marks - return marks.to_rev(mark) - -def export_ref(repo, name, kind, head): - global prefix, marks, mode - - ename = '%s/%s' % (kind, name) - try: - tip = marks.get_tip(ename) - tip = repo[tip].rev() - except: - tip = 0 - - revs = xrange(tip, head.rev() + 1) - total = len(revs) - - for rev in revs: - - c = repo[rev] - node = c.node() - - if marks.is_marked(c.hex()): - continue - - (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(node) - rev_branch = extra['branch'] - - author = "%s %d %s" % (fixup_user(user), time, gittz(tz)) - if 'committer' in extra: - user, time, tz = extra['committer'].rsplit(' ', 2) - committer = "%s %s %s" % (user, time, gittz(int(tz))) - else: - committer = author - - parents = [repo[p] for p in repo.changelog.parentrevs(rev) if p >= 0] - - if len(parents) == 0: - modified = c.manifest().keys() - removed = [] - else: - modified, removed = get_filechanges(repo, c, parents[0]) - - desc += '\n' - - if mode == 'hg': - extra_msg = '' - - if rev_branch != 'default': - extra_msg += 'branch : %s\n' % rev_branch - - renames = [] - for f in c.files(): - if f not in c.manifest(): - continue - rename = c.filectx(f).renamed() - if rename: - renames.append((rename[0], f)) - - for e in renames: - extra_msg += "rename : %s => %s\n" % e - - for key, value in extra.iteritems(): - if key in ('author', 'committer', 'encoding', 'message', 'branch', 'hg-git'): - continue - else: - extra_msg += "extra : %s : %s\n" % (key, urllib.quote(value)) - - if extra_msg: - desc += '\n--HG--\n' + extra_msg - - if len(parents) == 0 and rev: - print 'reset %s/%s' % (prefix, ename) - - modified_final = export_files(c.filectx(f) for f in modified) - - print "commit %s/%s" % (prefix, ename) - print "mark :%d" % (marks.get_mark(c.hex())) - print "author %s" % (author) - print "committer %s" % (committer) - print "data %d" % (len(desc)) - print desc - - if len(parents) > 0: - print "from :%s" % (rev_to_mark(parents[0])) - if len(parents) > 1: - print "merge :%s" % (rev_to_mark(parents[1])) - - for f in removed: - print "D %s" % (fix_file_path(f)) - for f in modified_final: - print "M %s :%u %s" % f - print - - progress = (rev - tip) - if (progress % 100 == 0): - print "progress revision %d '%s' (%d/%d)" % (rev, name, progress, total) - - # make sure the ref is updated - print "reset %s/%s" % (prefix, ename) - print "from :%u" % rev_to_mark(head) - print - - marks.set_tip(ename, head.hex()) - -def export_tag(repo, tag): - export_ref(repo, tag, 'tags', repo[hgref(tag)]) - -def export_bookmark(repo, bmark): - head = bmarks[hgref(bmark)] - export_ref(repo, bmark, 'bookmarks', head) - -def export_branch(repo, branch): - tip = get_branch_tip(repo, branch) - head = repo[tip] - export_ref(repo, branch, 'branches', head) - -def export_head(repo): - global g_head - export_ref(repo, g_head[0], 'bookmarks', g_head[1]) - -def do_capabilities(parser): - global prefix, dirname - - print "import" - print "export" - print "refspec refs/heads/branches/*:%s/branches/*" % prefix - print "refspec refs/heads/*:%s/bookmarks/*" % prefix - print "refspec refs/tags/*:%s/tags/*" % prefix - - path = os.path.join(dirname, 'marks-git') - - if os.path.exists(path): - print "*import-marks %s" % path - print "*export-marks %s" % path - print "option" - - print - -def branch_tip(branch): - return branches[branch][-1] - -def get_branch_tip(repo, branch): - global branches - - heads = branches.get(hgref(branch), None) - if not heads: - return None - - # verify there's only one head - if (len(heads) > 1): - warn("Branch '%s' has more than one head, consider merging" % branch) - return branch_tip(hgref(branch)) - - return heads[0] - -def list_head(repo, cur): - global g_head, bmarks, fake_bmark - - if 'default' not in branches: - # empty repo - return - - node = repo[branch_tip('default')] - head = 'master' if not 'master' in bmarks else 'default' - fake_bmark = head - bmarks[head] = node - - head = gitref(head) - print "@refs/heads/%s HEAD" % head - g_head = (head, node) - -def do_list(parser): - global branches, bmarks, track_branches - - repo = parser.repo - for bmark, node in bookmarks.listbookmarks(repo).iteritems(): - bmarks[bmark] = repo[node] - - cur = repo.dirstate.branch() - orig = peer if peer else repo - - for branch, heads in orig.branchmap().iteritems(): - # only open heads - heads = [h for h in heads if 'close' not in repo.changelog.read(h)[5]] - if heads: - branches[branch] = heads - - list_head(repo, cur) - - if track_branches: - for branch in branches: - print "? refs/heads/branches/%s" % gitref(branch) - - for bmark in bmarks: - print "? refs/heads/%s" % gitref(bmark) - - for tag, node in repo.tagslist(): - if tag == 'tip': - continue - print "? refs/tags/%s" % gitref(tag) - - print - -def do_import(parser): - repo = parser.repo - - path = os.path.join(dirname, 'marks-git') - - print "feature done" - if os.path.exists(path): - print "feature import-marks=%s" % path - print "feature export-marks=%s" % path - print "feature force" - sys.stdout.flush() - - tmp = encoding.encoding - encoding.encoding = 'utf-8' - - # lets get all the import lines - while parser.check('import'): - ref = parser[1] - - if (ref == 'HEAD'): - export_head(repo) - elif ref.startswith('refs/heads/branches/'): - branch = ref[len('refs/heads/branches/'):] - export_branch(repo, branch) - elif ref.startswith('refs/heads/'): - bmark = ref[len('refs/heads/'):] - export_bookmark(repo, bmark) - elif ref.startswith('refs/tags/'): - tag = ref[len('refs/tags/'):] - export_tag(repo, tag) - - parser.next() - - encoding.encoding = tmp - - print 'done' - -def parse_blob(parser): - global blob_marks - - parser.next() - mark = parser.get_mark() - parser.next() - data = parser.get_data() - blob_marks[mark] = data - parser.next() - -def get_merge_files(repo, p1, p2, files): - for e in repo[p1].files(): - if e not in files: - if e not in repo[p1].manifest(): - continue - f = { 'ctx' : repo[p1][e] } - files[e] = f - -def parse_commit(parser): - global marks, blob_marks, parsed_refs - global mode - - from_mark = merge_mark = None - - ref = parser[1] - parser.next() - - commit_mark = parser.get_mark() - parser.next() - author = parser.get_author() - parser.next() - committer = parser.get_author() - parser.next() - data = parser.get_data() - parser.next() - if parser.check('from'): - from_mark = parser.get_mark() - parser.next() - if parser.check('merge'): - merge_mark = parser.get_mark() - parser.next() - if parser.check('merge'): - die('octopus merges are not supported yet') - - # fast-export adds an extra newline - if data[-1] == '\n': - data = data[:-1] - - files = {} - - for line in parser: - if parser.check('M'): - t, m, mark_ref, path = line.split(' ', 3) - mark = int(mark_ref[1:]) - f = { 'mode' : hgmode(m), 'data' : blob_marks[mark] } - elif parser.check('D'): - t, path = line.split(' ', 1) - f = { 'deleted' : True } - else: - die('Unknown file command: %s' % line) - files[path] = f - - # only export the commits if we are on an internal proxy repo - if dry_run and not peer: - parsed_refs[ref] = None - return - - def getfilectx(repo, memctx, f): - of = files[f] - if 'deleted' in of: - raise IOError - if 'ctx' in of: - return of['ctx'] - is_exec = of['mode'] == 'x' - is_link = of['mode'] == 'l' - rename = of.get('rename', None) - return context.memfilectx(f, of['data'], - is_link, is_exec, rename) - - repo = parser.repo - - user, date, tz = author - extra = {} - - if committer != author: - extra['committer'] = "%s %u %u" % committer - - if from_mark: - p1 = mark_to_rev(from_mark) - else: - p1 = '0' * 40 - - if merge_mark: - p2 = mark_to_rev(merge_mark) - else: - p2 = '0' * 40 - - # - # If files changed from any of the parents, hg wants to know, but in git if - # nothing changed from the first parent, nothing changed. - # - if merge_mark: - get_merge_files(repo, p1, p2, files) - - # Check if the ref is supposed to be a named branch - if ref.startswith('refs/heads/branches/'): - branch = ref[len('refs/heads/branches/'):] - extra['branch'] = hgref(branch) - - if mode == 'hg': - i = data.find('\n--HG--\n') - if i >= 0: - tmp = data[i + len('\n--HG--\n'):].strip() - for k, v in [e.split(' : ', 1) for e in tmp.split('\n')]: - if k == 'rename': - old, new = v.split(' => ', 1) - files[new]['rename'] = old - elif k == 'branch': - extra[k] = v - elif k == 'extra': - ek, ev = v.split(' : ', 1) - extra[ek] = urllib.unquote(ev) - data = data[:i] - - ctx = context.memctx(repo, (p1, p2), data, - files.keys(), getfilectx, - user, (date, tz), extra) - - tmp = encoding.encoding - encoding.encoding = 'utf-8' - - node = hghex(repo.commitctx(ctx)) - - encoding.encoding = tmp - - parsed_refs[ref] = node - marks.new_mark(node, commit_mark) - -def parse_reset(parser): - global parsed_refs - - ref = parser[1] - parser.next() - # ugh - if parser.check('commit'): - parse_commit(parser) - return - if not parser.check('from'): - return - from_mark = parser.get_mark() - parser.next() - - try: - rev = mark_to_rev(from_mark) - except KeyError: - rev = None - parsed_refs[ref] = rev - -def parse_tag(parser): - name = parser[1] - parser.next() - from_mark = parser.get_mark() - parser.next() - tagger = parser.get_author() - parser.next() - data = parser.get_data() - parser.next() - - parsed_tags[name] = (tagger, data) - -def write_tag(repo, tag, node, msg, author): - branch = repo[node].branch() - tip = branch_tip(branch) - tip = repo[tip] - - def getfilectx(repo, memctx, f): - try: - fctx = tip.filectx(f) - data = fctx.data() - except error.ManifestLookupError: - data = "" - content = data + "%s %s\n" % (node, tag) - return context.memfilectx(f, content, False, False, None) - - p1 = tip.hex() - p2 = '0' * 40 - if author: - user, date, tz = author - date_tz = (date, tz) - else: - cmd = ['git', 'var', 'GIT_COMMITTER_IDENT'] - process = subprocess.Popen(cmd, stdout=subprocess.PIPE) - output, _ = process.communicate() - m = re.match('^.* <.*>', output) - if m: - user = m.group(0) - else: - user = repo.ui.username() - date_tz = None - - ctx = context.memctx(repo, (p1, p2), msg, - ['.hgtags'], getfilectx, - user, date_tz, {'branch' : branch}) - - tmp = encoding.encoding - encoding.encoding = 'utf-8' - - tagnode = repo.commitctx(ctx) - - encoding.encoding = tmp - - return (tagnode, branch) - -def checkheads_bmark(repo, ref, ctx): - bmark = ref[len('refs/heads/'):] - if not bmark in bmarks: - # new bmark - return True - - ctx_old = bmarks[bmark] - ctx_new = ctx - if not repo.changelog.descendant(ctx_old.rev(), ctx_new.rev()): - if force_push: - print "ok %s forced update" % ref - else: - print "error %s non-fast forward" % ref - return False - - return True - -def checkheads(repo, remote, p_revs): - - remotemap = remote.branchmap() - if not remotemap: - # empty repo - return True - - new = {} - ret = True - - for node, ref in p_revs.iteritems(): - ctx = repo[node] - branch = ctx.branch() - if not branch in remotemap: - # new branch - continue - if not ref.startswith('refs/heads/branches'): - if ref.startswith('refs/heads/'): - if not checkheads_bmark(repo, ref, ctx): - ret = False - - # only check branches - continue - new.setdefault(branch, []).append(ctx.rev()) - - for branch, heads in new.iteritems(): - old = [repo.changelog.rev(x) for x in remotemap[branch]] - for rev in heads: - if check_version(2, 3): - ancestors = repo.changelog.ancestors([rev], stoprev=min(old)) - else: - ancestors = repo.changelog.ancestors(rev) - found = False - - for x in old: - if x in ancestors: - found = True - break - - if found: - continue - - node = repo.changelog.node(rev) - ref = p_revs[node] - if force_push: - print "ok %s forced update" % ref - else: - print "error %s non-fast forward" % ref - ret = False - - return ret - -def push_unsafe(repo, remote, parsed_refs, p_revs): - - force = force_push - - fci = discovery.findcommonincoming - commoninc = fci(repo, remote, force=force) - common, _, remoteheads = commoninc - - if not checkheads(repo, remote, p_revs): - return None - - cg = repo.getbundle('push', heads=list(p_revs), common=common) - - unbundle = remote.capable('unbundle') - if unbundle: - if force: - remoteheads = ['force'] - return remote.unbundle(cg, remoteheads, 'push') - else: - return remote.addchangegroup(cg, 'push', repo.url()) - -def push(repo, remote, parsed_refs, p_revs): - if hasattr(remote, 'canpush') and not remote.canpush(): - print "error cannot push" - - if not p_revs: - # nothing to push - return - - lock = None - unbundle = remote.capable('unbundle') - if not unbundle: - lock = remote.lock() - try: - ret = push_unsafe(repo, remote, parsed_refs, p_revs) - finally: - if lock is not None: - lock.release() - - return ret - -def check_tip(ref, kind, name, heads): - try: - ename = '%s/%s' % (kind, name) - tip = marks.get_tip(ename) - except KeyError: - return True - else: - return tip in heads - -def do_export(parser): - global parsed_refs, bmarks, peer - - p_bmarks = [] - p_revs = {} - - parser.next() - - for line in parser.each_block('done'): - if parser.check('blob'): - parse_blob(parser) - elif parser.check('commit'): - parse_commit(parser) - elif parser.check('reset'): - parse_reset(parser) - elif parser.check('tag'): - parse_tag(parser) - elif parser.check('feature'): - pass - else: - die('unhandled export command: %s' % line) - - need_fetch = False - - for ref, node in parsed_refs.iteritems(): - bnode = hgbin(node) if node else None - if ref.startswith('refs/heads/branches'): - branch = ref[len('refs/heads/branches/'):] - if branch in branches and bnode in branches[branch]: - # up to date - continue - - if peer: - remotemap = peer.branchmap() - if remotemap and branch in remotemap: - heads = [hghex(e) for e in remotemap[branch]] - if not check_tip(ref, 'branches', branch, heads): - print "error %s fetch first" % ref - need_fetch = True - continue - - p_revs[bnode] = ref - print "ok %s" % ref - elif ref.startswith('refs/heads/'): - bmark = ref[len('refs/heads/'):] - new = node - old = bmarks[bmark].hex() if bmark in bmarks else '' - - if old == new: - continue - - print "ok %s" % ref - if bmark != fake_bmark and \ - not (bmark == 'master' and bmark not in parser.repo._bookmarks): - p_bmarks.append((ref, bmark, old, new)) - - if peer: - remote_old = peer.listkeys('bookmarks').get(bmark) - if remote_old: - if not check_tip(ref, 'bookmarks', bmark, remote_old): - print "error %s fetch first" % ref - need_fetch = True - continue - - p_revs[bnode] = ref - elif ref.startswith('refs/tags/'): - if dry_run: - print "ok %s" % ref - continue - tag = ref[len('refs/tags/'):] - tag = hgref(tag) - author, msg = parsed_tags.get(tag, (None, None)) - if mode == 'git': - if not msg: - msg = 'Added tag %s for changeset %s' % (tag, node[:12]); - tagnode, branch = write_tag(parser.repo, tag, node, msg, author) - p_revs[tagnode] = 'refs/heads/branches/' + gitref(branch) - else: - fp = parser.repo.opener('localtags', 'a') - fp.write('%s %s\n' % (node, tag)) - fp.close() - p_revs[bnode] = ref - print "ok %s" % ref - else: - # transport-helper/fast-export bugs - continue - - if need_fetch: - print - return - - if dry_run: - if peer and not force_push: - checkheads(parser.repo, peer, p_revs) - print - return - - if peer: - if not push(parser.repo, peer, parsed_refs, p_revs): - # do not update bookmarks - print - return - - # update remote bookmarks - remote_bmarks = peer.listkeys('bookmarks') - for ref, bmark, old, new in p_bmarks: - if force_push: - old = remote_bmarks.get(bmark, '') - if not peer.pushkey('bookmarks', bmark, old, new): - print "error %s" % ref - else: - # update local bookmarks - for ref, bmark, old, new in p_bmarks: - if not bookmarks.pushbookmark(parser.repo, bmark, old, new): - print "error %s" % ref - - print - -def do_option(parser): - global dry_run, force_push - _, key, value = parser.line.split(' ') - if key == 'dry-run': - dry_run = (value == 'true') - print 'ok' - elif key == 'force': - force_push = (value == 'true') - print 'ok' - else: - print 'unsupported' - -def fix_path(alias, repo, orig_url): - url = urlparse.urlparse(orig_url, 'file') - if url.scheme != 'file' or os.path.isabs(url.path): - return - abs_url = urlparse.urljoin("%s/" % os.getcwd(), orig_url) - cmd = ['git', 'config', 'remote.%s.url' % alias, "hg::%s" % abs_url] - subprocess.call(cmd) - -def main(args): - global prefix, gitdir, dirname, branches, bmarks - global marks, blob_marks, parsed_refs - global peer, mode, bad_mail, bad_name - global track_branches, force_push, is_tmp - global parsed_tags - global filenodes - global fake_bmark, hg_version - global dry_run - - alias = args[1] - url = args[2] - peer = None - - hg_git_compat = get_config_bool('remote-hg.hg-git-compat') - track_branches = get_config_bool('remote-hg.track-branches', True) - force_push = False - - if hg_git_compat: - mode = 'hg' - bad_mail = 'none@none' - bad_name = '' - else: - mode = 'git' - bad_mail = 'unknown' - bad_name = 'Unknown' - - if alias[4:] == url: - is_tmp = True - alias = hashlib.sha1(alias).hexdigest() - else: - is_tmp = False - - gitdir = os.environ['GIT_DIR'] - dirname = os.path.join(gitdir, 'hg', alias) - branches = {} - bmarks = {} - blob_marks = {} - parsed_refs = {} - marks = None - parsed_tags = {} - filenodes = {} - fake_bmark = None - try: - hg_version = tuple(int(e) for e in util.version().split('.')) - except: - hg_version = None - dry_run = False - - repo = get_repo(url, alias) - prefix = 'refs/hg/%s' % alias - - if not is_tmp: - fix_path(alias, peer or repo, url) - - marks_path = os.path.join(dirname, 'marks-hg') - marks = Marks(marks_path, repo) - - if sys.platform == 'win32': - import msvcrt - msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) - - parser = Parser(repo) - for line in parser: - if parser.check('capabilities'): - do_capabilities(parser) - elif parser.check('list'): - do_list(parser) - elif parser.check('import'): - do_import(parser) - elif parser.check('export'): - do_export(parser) - elif parser.check('option'): - do_option(parser) - else: - die('unhandled command: %s' % line) - sys.stdout.flush() -def bye(): - if not marks: - return - if not is_tmp: - marks.store() - else: - shutil.rmtree(dirname) +sys.stderr.write('WARNING: git-remote-hg is now maintained independently.\n') +sys.stderr.write('WARNING: For more information visit https://github.com/felipec/git-remote-hg\n') -atexit.register(bye) -sys.exit(main(sys.argv)) +sys.stderr.write('''WARNING: +WARNING: You can pick a directory on your $PATH and download it, e.g.: +WARNING: $ wget -O $HOME/bin/git-remote-hg \\ +WARNING: https://raw.github.com/felipec/git-remote-hg/master/git-remote-hg +WARNING: $ chmod +x $HOME/bin/git-remote-hg +''') diff --git a/contrib/remote-helpers/test-bzr.sh b/contrib/remote-helpers/test-bzr.sh deleted file mode 100755 index dce281f911..0000000000 --- a/contrib/remote-helpers/test-bzr.sh +++ /dev/null @@ -1,361 +0,0 @@ -#!/bin/sh -# -# Copyright (c) 2012 Felipe Contreras -# - -test_description='Test remote-bzr' - -. ./test-lib.sh - -if ! test_have_prereq PYTHON; then - skip_all='skipping remote-bzr tests; python not available' - test_done -fi - -if ! python -c 'import bzrlib'; then - skip_all='skipping remote-bzr tests; bzr not available' - test_done -fi - -check () { - echo $3 > expected && - git --git-dir=$1/.git log --format='%s' -1 $2 > actual - test_cmp expected actual -} - -bzr whoami "A U Thor <author@example.com>" - -test_expect_success 'cloning' ' - ( - bzr init bzrrepo && - cd bzrrepo && - echo one > content && - bzr add content && - bzr commit -m one - ) && - - git clone "bzr::bzrrepo" gitrepo && - check gitrepo HEAD one -' - -test_expect_success 'pulling' ' - ( - cd bzrrepo && - echo two > content && - bzr commit -m two - ) && - - (cd gitrepo && git pull) && - - check gitrepo HEAD two -' - -test_expect_success 'pushing' ' - ( - cd gitrepo && - echo three > content && - git commit -a -m three && - git push - ) && - - echo three > expected && - cat bzrrepo/content > actual && - test_cmp expected actual -' - -test_expect_success 'roundtrip' ' - ( - cd gitrepo && - git pull && - git log --format="%s" -1 origin/master > actual - ) && - echo three > expected && - test_cmp expected actual && - - (cd gitrepo && git push && git pull) && - - ( - cd bzrrepo && - echo four > content && - bzr commit -m four - ) && - - (cd gitrepo && git pull && git push) && - - check gitrepo HEAD four && - - ( - cd gitrepo && - echo five > content && - git commit -a -m five && - git push && git pull - ) && - - (cd bzrrepo && bzr revert) && - - echo five > expected && - cat bzrrepo/content > actual && - test_cmp expected actual -' - -cat > expected <<EOF -100644 blob 54f9d6da5c91d556e6b54340b1327573073030af content -100755 blob 68769579c3eaadbe555379b9c3538e6628bae1eb executable -120000 blob 6b584e8ece562ebffc15d38808cd6b98fc3d97ea link -EOF - -test_expect_success 'special modes' ' - ( - cd bzrrepo && - echo exec > executable - chmod +x executable && - bzr add executable - bzr commit -m exec && - ln -s content link - bzr add link - bzr commit -m link && - mkdir dir && - bzr add dir && - bzr commit -m dir - ) && - - ( - cd gitrepo && - git pull - git ls-tree HEAD > ../actual - ) && - - test_cmp expected actual && - - ( - cd gitrepo && - git cat-file -p HEAD:link > ../actual - ) && - - printf content > expected && - test_cmp expected actual -' - -cat > expected <<EOF -100644 blob 54f9d6da5c91d556e6b54340b1327573073030af content -100755 blob 68769579c3eaadbe555379b9c3538e6628bae1eb executable -120000 blob 6b584e8ece562ebffc15d38808cd6b98fc3d97ea link -040000 tree 35c0caa46693cef62247ac89a680f0c5ce32b37b movedir-new -EOF - -test_expect_success 'moving directory' ' - ( - cd bzrrepo && - mkdir movedir && - echo one > movedir/one && - echo two > movedir/two && - bzr add movedir && - bzr commit -m movedir && - bzr mv movedir movedir-new && - bzr commit -m movedir-new - ) && - - ( - cd gitrepo && - git pull && - git ls-tree HEAD > ../actual - ) && - - test_cmp expected actual -' - -test_expect_success 'different authors' ' - ( - cd bzrrepo && - echo john >> content && - bzr commit -m john \ - --author "Jane Rey <jrey@example.com>" \ - --author "John Doe <jdoe@example.com>" - ) && - - ( - cd gitrepo && - git pull && - git show --format="%an <%ae>, %cn <%ce>" --quiet > ../actual - ) && - - echo "Jane Rey <jrey@example.com>, A U Thor <author@example.com>" > expected && - test_cmp expected actual -' - -# cleanup previous stuff -rm -rf bzrrepo gitrepo - -test_expect_success 'fetch utf-8 filenames' ' - test_when_finished "rm -rf bzrrepo gitrepo && LC_ALL=C" && - - LC_ALL=en_US.UTF-8 - export LC_ALL - - ( - bzr init bzrrepo && - cd bzrrepo && - - echo test >> "ærø" && - bzr add "ærø" && - echo test >> "ø~?" && - bzr add "ø~?" && - bzr commit -m add-utf-8 && - echo test >> "ærø" && - bzr commit -m test-utf-8 && - bzr rm "ø~?" && - bzr mv "ærø" "ø~?" && - bzr commit -m bzr-mv-utf-8 - ) && - - ( - git clone "bzr::bzrrepo" gitrepo && - cd gitrepo && - git -c core.quotepath=false ls-files > ../actual - ) && - echo "ø~?" > expected && - test_cmp expected actual -' - -test_expect_success 'push utf-8 filenames' ' - test_when_finished "rm -rf bzrrepo gitrepo && LC_ALL=C" && - - mkdir -p tmp && cd tmp && - - LC_ALL=en_US.UTF-8 - export LC_ALL - - ( - bzr init bzrrepo && - cd bzrrepo && - - echo one >> content && - bzr add content && - bzr commit -m one - ) && - - ( - git clone "bzr::bzrrepo" gitrepo && - cd gitrepo && - - echo test >> "ærø" && - git add "ærø" && - git commit -m utf-8 && - - git push - ) && - - (cd bzrrepo && bzr ls > ../actual) && - printf "content\nærø\n" > expected && - test_cmp expected actual -' - -test_expect_success 'pushing a merge' ' - test_when_finished "rm -rf bzrrepo gitrepo" && - - ( - bzr init bzrrepo && - cd bzrrepo && - echo one > content && - bzr add content && - bzr commit -m one - ) && - - git clone "bzr::bzrrepo" gitrepo && - - ( - cd bzrrepo && - echo two > content && - bzr commit -m two - ) && - - ( - cd gitrepo && - echo three > content && - git commit -a -m three && - git fetch && - git merge origin/master || true && - echo three > content && - git commit -a --no-edit && - git push - ) && - - echo three > expected && - cat bzrrepo/content > actual && - test_cmp expected actual -' - -cat > expected <<EOF -origin/HEAD -origin/branch -origin/trunk -EOF - -test_expect_success 'proper bzr repo' ' - test_when_finished "rm -rf bzrrepo gitrepo" && - - bzr init-repo bzrrepo && - - ( - bzr init bzrrepo/trunk && - cd bzrrepo/trunk && - echo one >> content && - bzr add content && - bzr commit -m one - ) && - - ( - bzr branch bzrrepo/trunk bzrrepo/branch && - cd bzrrepo/branch && - echo two >> content && - bzr commit -m one - ) && - - ( - git clone "bzr::bzrrepo" gitrepo && - cd gitrepo && - git for-each-ref --format "%(refname:short)" refs/remotes/origin > ../actual - ) && - - test_cmp expected actual -' - -test_expect_success 'strip' ' - test_when_finished "rm -rf bzrrepo gitrepo" && - - ( - bzr init bzrrepo && - cd bzrrepo && - - echo one >> content && - bzr add content && - bzr commit -m one && - - echo two >> content && - bzr commit -m two - ) && - - git clone "bzr::bzrrepo" gitrepo && - - ( - cd bzrrepo && - bzr uncommit --force && - - echo three >> content && - bzr commit -m three && - - echo four >> content && - bzr commit -m four && - bzr log --line | sed -e "s/^[0-9][0-9]*: //" > ../expected - ) && - - ( - cd gitrepo && - git fetch && - git log --format="%an %ad %s" --date=short origin/master > ../actual - ) && - - test_cmp expected actual -' - -test_done diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh deleted file mode 100755 index f83d67d74f..0000000000 --- a/contrib/remote-helpers/test-hg-bidi.sh +++ /dev/null @@ -1,240 +0,0 @@ -#!/bin/sh -# -# Copyright (c) 2012 Felipe Contreras -# -# Base commands from hg-git tests: -# https://bitbucket.org/durin42/hg-git/src -# - -test_description='Test bidirectionality of remote-hg' - -. ./test-lib.sh - -if ! test_have_prereq PYTHON; then - skip_all='skipping remote-hg tests; python not available' - test_done -fi - -if ! python -c 'import mercurial'; then - skip_all='skipping remote-hg tests; mercurial not available' - test_done -fi - -# clone to a git repo -git_clone () { - git clone -q "hg::$1" $2 -} - -# clone to an hg repo -hg_clone () { - ( - hg init $2 && - cd $1 && - git push -q "hg::../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' - ) && - - (cd $2 && hg -q update) -} - -# push an hg repo -hg_push () { - ( - cd $2 - git checkout -q -b tmp && - git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' && - git checkout -q @{-1} && - git branch -q -D tmp 2> /dev/null || true - ) -} - -hg_log () { - hg -R $1 log --graph --debug -} - -setup () { - ( - echo "[ui]" - echo "username = A U Thor <author@example.com>" - echo "[defaults]" - echo "backout = -d \"0 0\"" - echo "commit = -d \"0 0\"" - echo "debugrawcommit = -d \"0 0\"" - echo "tag = -d \"0 0\"" - echo "[extensions]" - echo "graphlog =" - ) >> "$HOME"/.hgrc && - git config --global remote-hg.hg-git-compat true - git config --global remote-hg.track-branches true - - HGEDITOR=/usr/bin/true - GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230" - GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" - export HGEDITOR GIT_AUTHOR_DATE GIT_COMMITTER_DATE -} - -setup - -test_expect_success 'encoding' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - - echo alpha > alpha && - git add alpha && - git commit -m "add älphà " && - - GIT_AUTHOR_NAME="tést èncödîng" && - export GIT_AUTHOR_NAME && - echo beta > beta && - git add beta && - git commit -m "add beta" && - - echo gamma > gamma && - git add gamma && - git commit -m "add gämmâ" && - - : TODO git config i18n.commitencoding latin-1 && - echo delta > delta && - git add delta && - git commit -m "add déltà " - ) && - - hg_clone gitrepo hgrepo && - git_clone hgrepo gitrepo2 && - hg_clone gitrepo2 hgrepo2 && - - HGENCODING=utf-8 hg_log hgrepo > expected && - HGENCODING=utf-8 hg_log hgrepo2 > actual && - - test_cmp expected actual -' - -test_expect_success 'file removal' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - echo beta > beta && - git add beta && - git commit -m "add beta" - mkdir foo && - echo blah > foo/bar && - git add foo && - git commit -m "add foo" && - git rm alpha && - git commit -m "remove alpha" && - git rm foo/bar && - git commit -m "remove foo/bar" - ) && - - hg_clone gitrepo hgrepo && - git_clone hgrepo gitrepo2 && - hg_clone gitrepo2 hgrepo2 && - - hg_log hgrepo > expected && - hg_log hgrepo2 > actual && - - test_cmp expected actual -' - -test_expect_success 'git tags' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - git config receive.denyCurrentBranch ignore && - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - git tag alpha && - - echo beta > beta && - git add beta && - git commit -m "add beta" && - git tag -a -m "added tag beta" beta - ) && - - hg_clone gitrepo hgrepo && - git_clone hgrepo gitrepo2 && - hg_clone gitrepo2 hgrepo2 && - - hg_log hgrepo > expected && - hg_log hgrepo2 > actual && - - test_cmp expected actual -' - -test_expect_success 'hg branch' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - - echo alpha > alpha && - git add alpha && - git commit -q -m "add alpha" && - git checkout -q -b not-master - ) && - - ( - hg_clone gitrepo hgrepo && - - cd hgrepo && - hg -q co default && - hg mv alpha beta && - hg -q commit -m "rename alpha to beta" && - hg branch gamma | grep -v "permanent and global" && - hg -q commit -m "started branch gamma" - ) && - - hg_push hgrepo gitrepo && - hg_clone gitrepo hgrepo2 && - - : Back to the common revision && - (cd hgrepo && hg checkout default) && - - hg_log hgrepo > expected && - hg_log hgrepo2 > actual && - - test_cmp expected actual -' - -test_expect_success 'hg tags' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - git checkout -q -b not-master - ) && - - ( - hg_clone gitrepo hgrepo && - - cd hgrepo && - hg co default && - hg tag alpha - ) && - - hg_push hgrepo gitrepo && - hg_clone gitrepo hgrepo2 && - - hg_log hgrepo > expected && - hg_log hgrepo2 > actual && - - test_cmp expected actual -' - -test_done diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh deleted file mode 100755 index 2219284382..0000000000 --- a/contrib/remote-helpers/test-hg-hg-git.sh +++ /dev/null @@ -1,527 +0,0 @@ -#!/bin/sh -# -# Copyright (c) 2012 Felipe Contreras -# -# Base commands from hg-git tests: -# https://bitbucket.org/durin42/hg-git/src -# - -test_description='Test remote-hg output compared to hg-git' - -. ./test-lib.sh - -if ! test_have_prereq PYTHON; then - skip_all='skipping remote-hg tests; python not available' - test_done -fi - -if ! python -c 'import mercurial'; then - skip_all='skipping remote-hg tests; mercurial not available' - test_done -fi - -if ! python -c 'import hggit'; then - skip_all='skipping remote-hg tests; hg-git not available' - test_done -fi - -# clone to a git repo with git -git_clone_git () { - git clone -q "hg::$1" $2 && - (cd $2 && git checkout master && git branch -D default) -} - -# clone to an hg repo with git -hg_clone_git () { - ( - hg init $2 && - hg -R $2 bookmark -i master && - cd $1 && - git push -q "hg::../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' - ) && - - (cd $2 && hg -q update) -} - -# clone to a git repo with hg -git_clone_hg () { - ( - git init -q $2 && - cd $1 && - hg bookmark -i -f -r tip master && - hg -q push -r master ../$2 || true - ) -} - -# clone to an hg repo with hg -hg_clone_hg () { - hg -q clone $1 $2 -} - -# push an hg repo with git -hg_push_git () { - ( - cd $2 - git checkout -q -b tmp && - git fetch -q "hg::../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' && - git branch -D default && - git checkout -q @{-1} && - git branch -q -D tmp 2> /dev/null || true - ) -} - -# push an hg git repo with hg -hg_push_hg () { - ( - cd $1 && - hg -q push ../$2 || true - ) -} - -hg_log () { - hg -R $1 log --graph --debug >log && - grep -v 'tag: *default/' log -} - -git_log () { - git --git-dir=$1/.git fast-export --branches -} - -setup () { - ( - echo "[ui]" - echo "username = A U Thor <author@example.com>" - echo "[defaults]" - echo "backout = -d \"0 0\"" - echo "commit = -d \"0 0\"" - echo "debugrawcommit = -d \"0 0\"" - echo "tag = -d \"0 0\"" - echo "[extensions]" - echo "hgext.bookmarks =" - echo "hggit =" - echo "graphlog =" - ) >> "$HOME"/.hgrc && - git config --global receive.denycurrentbranch warn - git config --global remote-hg.hg-git-compat true - git config --global remote-hg.track-branches false - - HGEDITOR=true - HGMERGE=true - - GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230" - GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" - export HGEDITOR HGMERGE GIT_AUTHOR_DATE GIT_COMMITTER_DATE -} - -setup - -test_expect_success 'executable bit' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - echo alpha > alpha && - chmod 0644 alpha && - git add alpha && - git commit -m "add alpha" && - chmod 0755 alpha && - git add alpha && - git commit -m "set executable bit" && - chmod 0644 alpha && - git add alpha && - git commit -m "clear executable bit" - ) && - - for x in hg git; do - ( - hg_clone_$x gitrepo hgrepo-$x && - cd hgrepo-$x && - hg_log . && - hg manifest -r 1 -v && - hg manifest -v - ) > output-$x && - - git_clone_$x hgrepo-$x gitrepo2-$x && - git_log gitrepo2-$x > log-$x - done && - - test_cmp output-hg output-git && - test_cmp log-hg log-git -' - -test_expect_success 'symlink' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - ln -s alpha beta && - git add beta && - git commit -m "add beta" - ) && - - for x in hg git; do - ( - hg_clone_$x gitrepo hgrepo-$x && - cd hgrepo-$x && - hg_log . && - hg manifest -v - ) > output-$x && - - git_clone_$x hgrepo-$x gitrepo2-$x && - git_log gitrepo2-$x > log-$x - done && - - test_cmp output-hg output-git && - test_cmp log-hg log-git -' - -test_expect_success 'merge conflict 1' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - hg init hgrepo1 && - cd hgrepo1 && - echo A > afile && - hg add afile && - hg ci -m "origin" && - - echo B > afile && - hg ci -m "A->B" && - - hg up -r0 && - echo C > afile && - hg ci -m "A->C" && - - hg merge -r1 && - echo C > afile && - hg resolve -m afile && - hg ci -m "merge to C" - ) && - - for x in hg git; do - git_clone_$x hgrepo1 gitrepo-$x && - hg_clone_$x gitrepo-$x hgrepo2-$x && - hg_log hgrepo2-$x > hg-log-$x && - git_log gitrepo-$x > git-log-$x - done && - - test_cmp hg-log-hg hg-log-git && - test_cmp git-log-hg git-log-git -' - -test_expect_success 'merge conflict 2' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - hg init hgrepo1 && - cd hgrepo1 && - echo A > afile && - hg add afile && - hg ci -m "origin" && - - echo B > afile && - hg ci -m "A->B" && - - hg up -r0 && - echo C > afile && - hg ci -m "A->C" && - - hg merge -r1 || true && - echo B > afile && - hg resolve -m afile && - hg ci -m "merge to B" - ) && - - for x in hg git; do - git_clone_$x hgrepo1 gitrepo-$x && - hg_clone_$x gitrepo-$x hgrepo2-$x && - hg_log hgrepo2-$x > hg-log-$x && - git_log gitrepo-$x > git-log-$x - done && - - test_cmp hg-log-hg hg-log-git && - test_cmp git-log-hg git-log-git -' - -test_expect_success 'converged merge' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - hg init hgrepo1 && - cd hgrepo1 && - echo A > afile && - hg add afile && - hg ci -m "origin" && - - echo B > afile && - hg ci -m "A->B" && - - echo C > afile && - hg ci -m "B->C" && - - hg up -r0 && - echo C > afile && - hg ci -m "A->C" && - - hg merge -r2 || true && - hg ci -m "merge" - ) && - - for x in hg git; do - git_clone_$x hgrepo1 gitrepo-$x && - hg_clone_$x gitrepo-$x hgrepo2-$x && - hg_log hgrepo2-$x > hg-log-$x && - git_log gitrepo-$x > git-log-$x - done && - - test_cmp hg-log-hg hg-log-git && - test_cmp git-log-hg git-log-git -' - -test_expect_success 'encoding' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - - echo alpha > alpha && - git add alpha && - git commit -m "add älphà " && - - GIT_AUTHOR_NAME="tést èncödîng" && - export GIT_AUTHOR_NAME && - echo beta > beta && - git add beta && - git commit -m "add beta" && - - echo gamma > gamma && - git add gamma && - git commit -m "add gämmâ" && - - : TODO git config i18n.commitencoding latin-1 && - echo delta > delta && - git add delta && - git commit -m "add déltà " - ) && - - for x in hg git; do - hg_clone_$x gitrepo hgrepo-$x && - git_clone_$x hgrepo-$x gitrepo2-$x && - - HGENCODING=utf-8 hg_log hgrepo-$x > hg-log-$x && - git_log gitrepo2-$x > git-log-$x - done && - - test_cmp hg-log-hg hg-log-git && - test_cmp git-log-hg git-log-git -' - -test_expect_success 'file removal' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - echo beta > beta && - git add beta && - git commit -m "add beta" - mkdir foo && - echo blah > foo/bar && - git add foo && - git commit -m "add foo" && - git rm alpha && - git commit -m "remove alpha" && - git rm foo/bar && - git commit -m "remove foo/bar" - ) && - - for x in hg git; do - ( - hg_clone_$x gitrepo hgrepo-$x && - cd hgrepo-$x && - hg_log . && - hg manifest -r 3 && - hg manifest - ) > output-$x && - - git_clone_$x hgrepo-$x gitrepo2-$x && - git_log gitrepo2-$x > log-$x - done && - - test_cmp output-hg output-git && - test_cmp log-hg log-git -' - -test_expect_success 'git tags' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - ( - git init -q gitrepo && - cd gitrepo && - git config receive.denyCurrentBranch ignore && - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - git tag alpha && - - echo beta > beta && - git add beta && - git commit -m "add beta" && - git tag -a -m "added tag beta" beta - ) && - - for x in hg git; do - hg_clone_$x gitrepo hgrepo-$x && - hg_log hgrepo-$x > log-$x - done && - - test_cmp log-hg log-git -' - -test_expect_success 'hg author' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - for x in hg git; do - ( - git init -q gitrepo-$x && - cd gitrepo-$x && - - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - git checkout -q -b not-master - ) && - - ( - hg_clone_$x gitrepo-$x hgrepo-$x && - cd hgrepo-$x && - - hg co master && - echo beta > beta && - hg add beta && - hg commit -u "test" -m "add beta" && - - echo gamma >> beta && - hg commit -u "test <test@example.com> (comment)" -m "modify beta" && - - echo gamma > gamma && - hg add gamma && - hg commit -u "<test@example.com>" -m "add gamma" && - - echo delta > delta && - hg add delta && - hg commit -u "name<test@example.com>" -m "add delta" && - - echo epsilon > epsilon && - hg add epsilon && - hg commit -u "name <test@example.com" -m "add epsilon" && - - echo zeta > zeta && - hg add zeta && - hg commit -u " test " -m "add zeta" && - - echo eta > eta && - hg add eta && - hg commit -u "test < test@example.com >" -m "add eta" && - - echo theta > theta && - hg add theta && - hg commit -u "test >test@example.com>" -m "add theta" && - - echo iota > iota && - hg add iota && - hg commit -u "test <test <at> example <dot> com>" -m "add iota" - ) && - - hg_push_$x hgrepo-$x gitrepo-$x && - hg_clone_$x gitrepo-$x hgrepo2-$x && - - hg_log hgrepo2-$x > hg-log-$x && - git_log gitrepo-$x > git-log-$x - done && - - test_cmp hg-log-hg hg-log-git && - test_cmp git-log-hg git-log-git -' - -test_expect_success 'hg branch' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - for x in hg git; do - ( - git init -q gitrepo-$x && - cd gitrepo-$x && - - echo alpha > alpha && - git add alpha && - git commit -q -m "add alpha" && - git checkout -q -b not-master - ) && - - ( - hg_clone_$x gitrepo-$x hgrepo-$x && - - cd hgrepo-$x && - hg -q co master && - hg mv alpha beta && - hg -q commit -m "rename alpha to beta" && - hg branch gamma | grep -v "permanent and global" && - hg -q commit -m "started branch gamma" - ) && - - hg_push_$x hgrepo-$x gitrepo-$x && - hg_clone_$x gitrepo-$x hgrepo2-$x && - - hg_log hgrepo2-$x > hg-log-$x && - git_log gitrepo-$x > git-log-$x - done && - - test_cmp hg-log-hg hg-log-git && - test_cmp git-log-hg git-log-git -' - -test_expect_success 'hg tags' ' - test_when_finished "rm -rf gitrepo* hgrepo*" && - - for x in hg git; do - ( - git init -q gitrepo-$x && - cd gitrepo-$x && - - echo alpha > alpha && - git add alpha && - git commit -m "add alpha" && - git checkout -q -b not-master - ) && - - ( - hg_clone_$x gitrepo-$x hgrepo-$x && - - cd hgrepo-$x && - hg co master && - hg tag alpha - ) && - - hg_push_$x hgrepo-$x gitrepo-$x && - hg_clone_$x gitrepo-$x hgrepo2-$x && - - ( - git --git-dir=gitrepo-$x/.git tag -l && - hg_log hgrepo2-$x && - cat hgrepo2-$x/.hgtags - ) > output-$x - done && - - test_cmp output-hg output-git -' - -test_done diff --git a/contrib/remote-helpers/test-hg.sh b/contrib/remote-helpers/test-hg.sh deleted file mode 100755 index f7ce8aa853..0000000000 --- a/contrib/remote-helpers/test-hg.sh +++ /dev/null @@ -1,692 +0,0 @@ -#!/bin/sh -# -# Copyright (c) 2012 Felipe Contreras -# -# Base commands from hg-git tests: -# https://bitbucket.org/durin42/hg-git/src -# - -test_description='Test remote-hg' - -. ./test-lib.sh - -if ! test_have_prereq PYTHON; then - skip_all='skipping remote-hg tests; python not available' - test_done -fi - -if ! python -c 'import mercurial'; then - skip_all='skipping remote-hg tests; mercurial not available' - test_done -fi - -check () { - echo $3 > expected && - git --git-dir=$1/.git log --format='%s' -1 $2 > actual - test_cmp expected actual -} - -check_branch () { - if [ -n "$3" ]; then - echo $3 > expected && - hg -R $1 log -r $2 --template '{desc}\n' > actual && - test_cmp expected actual - else - hg -R $1 branches > out && - ! grep $2 out - fi -} - -check_bookmark () { - if [ -n "$3" ]; then - echo $3 > expected && - hg -R $1 log -r "bookmark('$2')" --template '{desc}\n' > actual && - test_cmp expected actual - else - hg -R $1 bookmarks > out && - ! grep $2 out - fi -} - -check_push () { - local expected_ret=$1 ret=0 ref_ret=0 IFS=':' - - shift - git push origin "$@" 2> error - ret=$? - cat error - - while read branch kind - do - case "$kind" in - 'new') - grep "^ \* \[new branch\] *${branch} -> ${branch}$" error || ref_ret=1 - ;; - 'non-fast-forward') - grep "^ ! \[rejected\] *${branch} -> ${branch} (non-fast-forward)$" error || ref_ret=1 - ;; - 'fetch-first') - grep "^ ! \[rejected\] *${branch} -> ${branch} (fetch first)$" error || ref_ret=1 - ;; - 'forced-update') - grep "^ + [a-f0-9]*\.\.\.[a-f0-9]* *${branch} -> ${branch} (forced update)$" error || ref_ret=1 - ;; - '') - grep "^ [a-f0-9]*\.\.[a-f0-9]* *${branch} -> ${branch}$" error || ref_ret=1 - ;; - esac - let 'ref_ret' && echo "match for '$branch' failed" && break - done - - if let 'expected_ret != ret || ref_ret' - then - return 1 - fi - - return 0 -} - -setup () { - ( - echo "[ui]" - echo "username = H G Wells <wells@example.com>" - echo "[extensions]" - echo "mq =" - ) >> "$HOME"/.hgrc && - - GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230" && - GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" && - export GIT_COMMITTER_DATE GIT_AUTHOR_DATE -} - -setup - -test_expect_success 'cloning' ' - test_when_finished "rm -rf gitrepo*" && - - ( - hg init hgrepo && - cd hgrepo && - echo zero > content && - hg add content && - hg commit -m zero - ) && - - git clone "hg::hgrepo" gitrepo && - check gitrepo HEAD zero -' - -test_expect_success 'cloning with branches' ' - test_when_finished "rm -rf gitrepo*" && - - ( - cd hgrepo && - hg branch next && - echo next > content && - hg commit -m next - ) && - - git clone "hg::hgrepo" gitrepo && - check gitrepo origin/branches/next next -' - -test_expect_success 'cloning with bookmarks' ' - test_when_finished "rm -rf gitrepo*" && - - ( - cd hgrepo && - hg checkout default && - hg bookmark feature-a && - echo feature-a > content && - hg commit -m feature-a - ) && - - git clone "hg::hgrepo" gitrepo && - check gitrepo origin/feature-a feature-a -' - -test_expect_success 'update bookmark' ' - test_when_finished "rm -rf gitrepo*" && - - ( - cd hgrepo && - hg bookmark devel - ) && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - git checkout --quiet devel && - echo devel > content && - git commit -a -m devel && - git push --quiet - ) && - - check_bookmark hgrepo devel devel -' - -test_expect_success 'new bookmark' ' - test_when_finished "rm -rf gitrepo*" && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - git checkout --quiet -b feature-b && - echo feature-b > content && - git commit -a -m feature-b && - git push --quiet origin feature-b - ) && - - check_bookmark hgrepo feature-b feature-b -' - -# cleanup previous stuff -rm -rf hgrepo - -author_test () { - echo $1 >> content && - hg commit -u "$2" -m "add $1" && - echo "$3" >> ../expected -} - -test_expect_success 'authors' ' - test_when_finished "rm -rf hgrepo gitrepo" && - - ( - hg init hgrepo && - cd hgrepo && - - touch content && - hg add content && - - > ../expected && - author_test alpha "" "H G Wells <wells@example.com>" && - author_test beta "test" "test <unknown>" && - author_test beta "test <test@example.com> (comment)" "test <test@example.com>" && - author_test gamma "<test@example.com>" "Unknown <test@example.com>" && - author_test delta "name<test@example.com>" "name <test@example.com>" && - author_test epsilon "name <test@example.com" "name <test@example.com>" && - author_test zeta " test " "test <unknown>" && - author_test eta "test < test@example.com >" "test <test@example.com>" && - author_test theta "test >test@example.com>" "test <test@example.com>" && - author_test iota "test < test <at> example <dot> com>" "test <unknown>" && - author_test kappa "test@example.com" "Unknown <test@example.com>" - ) && - - git clone "hg::hgrepo" gitrepo && - git --git-dir=gitrepo/.git log --reverse --format="%an <%ae>" > actual && - - test_cmp expected actual -' - -test_expect_success 'strip' ' - test_when_finished "rm -rf hgrepo gitrepo" && - - ( - hg init hgrepo && - cd hgrepo && - - echo one >> content && - hg add content && - hg commit -m one && - - echo two >> content && - hg commit -m two - ) && - - git clone "hg::hgrepo" gitrepo && - - ( - cd hgrepo && - hg strip 1 && - - echo three >> content && - hg commit -m three && - - echo four >> content && - hg commit -m four - ) && - - ( - cd gitrepo && - git fetch && - git log --format="%s" origin/master > ../actual - ) && - - hg -R hgrepo log --template "{desc}\n" > expected && - test_cmp actual expected -' - -test_expect_success 'remote push with master bookmark' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - ( - hg init hgrepo && - cd hgrepo && - echo zero > content && - hg add content && - hg commit -m zero && - hg bookmark master && - echo one > content && - hg commit -m one - ) && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - echo two > content && - git commit -a -m two && - git push - ) && - - check_branch hgrepo default two -' - -cat > expected <<EOF -changeset: 0:6e2126489d3d -tag: tip -user: A U Thor <author@example.com> -date: Mon Jan 01 00:00:00 2007 +0230 -summary: one - -EOF - -test_expect_success 'remote push from master branch' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - hg init hgrepo && - - ( - git init gitrepo && - cd gitrepo && - git remote add origin "hg::../hgrepo" && - echo one > content && - git add content && - git commit -a -m one && - git push origin master - ) && - - hg -R hgrepo log > actual && - cat actual && - test_cmp expected actual && - - check_branch hgrepo default one -' - -GIT_REMOTE_HG_TEST_REMOTE=1 -export GIT_REMOTE_HG_TEST_REMOTE - -test_expect_success 'remote cloning' ' - test_when_finished "rm -rf gitrepo*" && - - ( - hg init hgrepo && - cd hgrepo && - echo zero > content && - hg add content && - hg commit -m zero - ) && - - git clone "hg::hgrepo" gitrepo && - check gitrepo HEAD zero -' - -test_expect_success 'remote update bookmark' ' - test_when_finished "rm -rf gitrepo*" && - - ( - cd hgrepo && - hg bookmark devel - ) && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - git checkout --quiet devel && - echo devel > content && - git commit -a -m devel && - git push --quiet - ) && - - check_bookmark hgrepo devel devel -' - -test_expect_success 'remote new bookmark' ' - test_when_finished "rm -rf gitrepo*" && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - git checkout --quiet -b feature-b && - echo feature-b > content && - git commit -a -m feature-b && - git push --quiet origin feature-b - ) && - - check_bookmark hgrepo feature-b feature-b -' - -test_expect_success 'remote push diverged' ' - test_when_finished "rm -rf gitrepo*" && - - git clone "hg::hgrepo" gitrepo && - - ( - cd hgrepo && - hg checkout default && - echo bump > content && - hg commit -m bump - ) && - - ( - cd gitrepo && - echo diverge > content && - git commit -a -m diverged && - check_push 1 <<-EOF - master:non-fast-forward - EOF - ) && - - check_branch hgrepo default bump -' - -test_expect_success 'remote update bookmark diverge' ' - test_when_finished "rm -rf gitrepo*" && - - ( - cd hgrepo && - hg checkout tip^ && - hg bookmark diverge - ) && - - git clone "hg::hgrepo" gitrepo && - - ( - cd hgrepo && - echo "bump bookmark" > content && - hg commit -m "bump bookmark" - ) && - - ( - cd gitrepo && - git checkout --quiet diverge && - echo diverge > content && - git commit -a -m diverge && - check_push 1 <<-EOF - diverge:fetch-first - EOF - ) && - - check_bookmark hgrepo diverge "bump bookmark" -' - -test_expect_success 'remote new bookmark multiple branch head' ' - test_when_finished "rm -rf gitrepo*" && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - git checkout --quiet -b feature-c HEAD^ && - echo feature-c > content && - git commit -a -m feature-c && - git push --quiet origin feature-c - ) && - - check_bookmark hgrepo feature-c feature-c -' - -# cleanup previous stuff -rm -rf hgrepo - -setup_big_push () { - ( - hg init hgrepo && - cd hgrepo && - echo zero > content && - hg add content && - hg commit -m zero && - hg bookmark bad_bmark1 && - echo one > content && - hg commit -m one && - hg bookmark bad_bmark2 && - hg bookmark good_bmark && - hg bookmark -i good_bmark && - hg -q branch good_branch && - echo "good branch" > content && - hg commit -m "good branch" && - hg -q branch bad_branch && - echo "bad branch" > content && - hg commit -m "bad branch" - ) && - - git clone "hg::hgrepo" gitrepo && - - ( - cd gitrepo && - echo two > content && - git commit -q -a -m two && - - git checkout -q good_bmark && - echo three > content && - git commit -q -a -m three && - - git checkout -q bad_bmark1 && - git reset --hard HEAD^ && - echo four > content && - git commit -q -a -m four && - - git checkout -q bad_bmark2 && - git reset --hard HEAD^ && - echo five > content && - git commit -q -a -m five && - - git checkout -q -b new_bmark master && - echo six > content && - git commit -q -a -m six && - - git checkout -q branches/good_branch && - echo seven > content && - git commit -q -a -m seven && - echo eight > content && - git commit -q -a -m eight && - - git checkout -q branches/bad_branch && - git reset --hard HEAD^ && - echo nine > content && - git commit -q -a -m nine && - - git checkout -q -b branches/new_branch master && - echo ten > content && - git commit -q -a -m ten - ) -} - -test_expect_success 'remote big push' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - setup_big_push - - ( - cd gitrepo && - - check_push 1 --all <<-EOF - master - good_bmark - branches/good_branch - new_bmark:new - branches/new_branch:new - bad_bmark1:non-fast-forward - bad_bmark2:non-fast-forward - branches/bad_branch:non-fast-forward - EOF - ) && - - check_branch hgrepo default one && - check_branch hgrepo good_branch "good branch" && - check_branch hgrepo bad_branch "bad branch" && - check_branch hgrepo new_branch '' && - check_bookmark hgrepo good_bmark one && - check_bookmark hgrepo bad_bmark1 one && - check_bookmark hgrepo bad_bmark2 one && - check_bookmark hgrepo new_bmark '' -' - -test_expect_success 'remote big push fetch first' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - ( - hg init hgrepo && - cd hgrepo && - echo zero > content && - hg add content && - hg commit -m zero && - hg bookmark bad_bmark && - hg bookmark good_bmark && - hg bookmark -i good_bmark && - hg -q branch good_branch && - echo "good branch" > content && - hg commit -m "good branch" && - hg -q branch bad_branch && - echo "bad branch" > content && - hg commit -m "bad branch" - ) && - - git clone "hg::hgrepo" gitrepo && - - ( - cd hgrepo && - hg bookmark -f bad_bmark && - echo update_bmark > content && - hg commit -m "update bmark" - ) && - - ( - cd gitrepo && - echo two > content && - git commit -q -a -m two && - - git checkout -q good_bmark && - echo three > content && - git commit -q -a -m three && - - git checkout -q bad_bmark && - echo four > content && - git commit -q -a -m four && - - git checkout -q branches/bad_branch && - echo five > content && - git commit -q -a -m five && - - check_push 1 --all <<-EOF - master - good_bmark - new_bmark:new - new_branch:new - bad_bmark:fetch-first - branches/bad_branch:festch-first - EOF - - git fetch && - - check_push 1 --all <<-EOF - master - good_bmark - bad_bmark:non-fast-forward - branches/bad_branch:non-fast-forward - EOF - ) -' - -test_expect_failure 'remote big push force' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - setup_big_push - - ( - cd gitrepo && - - check_push 0 --force --all <<-EOF - master - good_bmark - branches/good_branch - new_bmark:new - branches/new_branch:new - bad_bmark1:forced-update - bad_bmark2:forced-update - branches/bad_branch:forced-update - EOF - ) && - - check_branch hgrepo default six && - check_branch hgrepo good_branch eight && - check_branch hgrepo bad_branch nine && - check_branch hgrepo new_branch ten && - check_bookmark hgrepo good_bmark three && - check_bookmark hgrepo bad_bmark1 four && - check_bookmark hgrepo bad_bmark2 five && - check_bookmark hgrepo new_bmark six -' - -test_expect_failure 'remote big push dry-run' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - setup_big_push - - ( - cd gitrepo && - - check_push 0 --dry-run --all <<-EOF - master - good_bmark - branches/good_branch - new_bmark:new - branches/new_branch:new - bad_bmark1:non-fast-forward - bad_bmark2:non-fast-forward - branches/bad_branch:non-fast-forward - EOF - - check_push 0 --dry-run master good_bmark new_bmark branches/good_branch branches/new_branch <<-EOF - master - good_bmark - branches/good_branch - new_bmark:new - branches/new_branch:new - EOF - ) && - - check_branch hgrepo default one && - check_branch hgrepo good_branch "good branch" && - check_branch hgrepo bad_branch "bad branch" && - check_branch hgrepo new_branch '' && - check_bookmark hgrepo good_bmark one && - check_bookmark hgrepo bad_bmark1 one && - check_bookmark hgrepo bad_bmark2 one && - check_bookmark hgrepo new_bmark '' -' - -test_expect_success 'remote double failed push' ' - test_when_finished "rm -rf hgrepo gitrepo*" && - - ( - hg init hgrepo && - cd hgrepo && - echo zero > content && - hg add content && - hg commit -m zero && - echo one > content && - hg commit -m one - ) && - - ( - git clone "hg::hgrepo" gitrepo && - cd gitrepo && - git reset --hard HEAD^ && - echo two > content && - git commit -a -m two && - test_expect_code 1 git push && - test_expect_code 1 git push - ) -' - -test_done diff --git a/contrib/subtree/.gitignore b/contrib/subtree/.gitignore index 91360a3d7f..0b9381abca 100644 --- a/contrib/subtree/.gitignore +++ b/contrib/subtree/.gitignore @@ -1,6 +1,7 @@ *~ git-subtree -git-subtree.xml git-subtree.1 +git-subtree.html +git-subtree.xml mainline subproj diff --git a/contrib/subtree/Makefile b/contrib/subtree/Makefile index b50750565f..3071baf493 100644 --- a/contrib/subtree/Makefile +++ b/contrib/subtree/Makefile @@ -1,19 +1,34 @@ +# The default target of this Makefile is... +all:: + -include ../../config.mak.autogen -include ../../config.mak prefix ?= /usr/local +gitexecdir ?= $(prefix)/libexec/git-core mandir ?= $(prefix)/share/man -libexecdir ?= $(prefix)/libexec/git-core -gitdir ?= $(shell git --exec-path) man1dir ?= $(mandir)/man1 +htmldir ?= $(prefix)/share/doc/git-doc + +../../GIT-VERSION-FILE: FORCE + $(MAKE) -C ../../ GIT-VERSION-FILE -gitver ?= $(word 3,$(shell git --version)) +-include ../../GIT-VERSION-FILE # this should be set to a 'standard' bsd-type install program -INSTALL ?= install +INSTALL ?= install +RM ?= rm -f + +ASCIIDOC = asciidoc +XMLTO = xmlto + +ifndef SHELL_PATH + SHELL_PATH = /bin/sh +endif +SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH)) -ASCIIDOC_CONF = ../../Documentation/asciidoc.conf -MANPAGE_NORMAL_XSL = ../../Documentation/manpage-normal.xsl +ASCIIDOC_CONF = ../../Documentation/asciidoc.conf +MANPAGE_XSL = ../../Documentation/manpage-normal.xsl GIT_SUBTREE_SH := git-subtree.sh GIT_SUBTREE := git-subtree @@ -21,33 +36,46 @@ GIT_SUBTREE := git-subtree GIT_SUBTREE_DOC := git-subtree.1 GIT_SUBTREE_XML := git-subtree.xml GIT_SUBTREE_TXT := git-subtree.txt +GIT_SUBTREE_HTML := git-subtree.html -all: $(GIT_SUBTREE) +all:: $(GIT_SUBTREE) $(GIT_SUBTREE): $(GIT_SUBTREE_SH) - cp $< $@ && chmod +x $@ + sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' $< >$@ + chmod +x $@ -doc: $(GIT_SUBTREE_DOC) +doc: $(GIT_SUBTREE_DOC) $(GIT_SUBTREE_HTML) install: $(GIT_SUBTREE) - $(INSTALL) -m 755 $(GIT_SUBTREE) $(DESTDIR)$(libexecdir) + $(INSTALL) -d -m 755 $(DESTDIR)$(gitexecdir) + $(INSTALL) -m 755 $(GIT_SUBTREE) $(DESTDIR)$(gitexecdir) -install-doc: install-man +install-doc: install-man install-html install-man: $(GIT_SUBTREE_DOC) $(INSTALL) -d -m 755 $(DESTDIR)$(man1dir) $(INSTALL) -m 644 $^ $(DESTDIR)$(man1dir) +install-html: $(GIT_SUBTREE_HTML) + $(INSTALL) -d -m 755 $(DESTDIR)$(htmldir) + $(INSTALL) -m 644 $^ $(DESTDIR)$(htmldir) + $(GIT_SUBTREE_DOC): $(GIT_SUBTREE_XML) - xmlto -m $(MANPAGE_NORMAL_XSL) man $^ + $(XMLTO) -m $(MANPAGE_XSL) man $^ $(GIT_SUBTREE_XML): $(GIT_SUBTREE_TXT) - asciidoc -b docbook -d manpage -f $(ASCIIDOC_CONF) \ - -agit_version=$(gitver) $^ + $(ASCIIDOC) -b docbook -d manpage -f $(ASCIIDOC_CONF) \ + -agit_version=$(GIT_VERSION) $^ + +$(GIT_SUBTREE_HTML): $(GIT_SUBTREE_TXT) + $(ASCIIDOC) -b xhtml11 -d manpage -f $(ASCIIDOC_CONF) \ + -agit_version=$(GIT_VERSION) $^ test: $(MAKE) -C t/ test clean: - rm -f *~ *.xml *.html *.1 - rm -rf subproj mainline + $(RM) $(GIT_SUBTREE) + $(RM) *.xml *.html *.1 + +.PHONY: FORCE diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh index 51ae932e5e..9f06571851 100755 --- a/contrib/subtree/git-subtree.sh +++ b/contrib/subtree/git-subtree.sh @@ -9,10 +9,10 @@ if [ $# -eq 0 ]; then fi OPTS_SPEC="\ git subtree add --prefix=<prefix> <commit> -git subtree add --prefix=<prefix> <repository> <commit> +git subtree add --prefix=<prefix> <repository> <ref> git subtree merge --prefix=<prefix> <commit> -git subtree pull --prefix=<prefix> <repository> <refspec...> -git subtree push --prefix=<prefix> <repository> <refspec...> +git subtree pull --prefix=<prefix> <repository> <ref> +git subtree push --prefix=<prefix> <repository> <ref> git subtree split --prefix=<prefix> <commit...> -- h,help show the help @@ -26,7 +26,7 @@ b,branch= create a new branch from the split subtree ignore-joins ignore prior --rejoin commits onto= try connecting new tree to an existing one rejoin merge the new branch back into HEAD - options for 'add', 'merge', 'pull' and 'push' + options for 'add', 'merge', and 'pull' squash merge subtree changes as a single commit " eval "$(echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?)" @@ -46,18 +46,26 @@ ignore_joins= annotate= squash= message= +prefix= debug() { if [ -n "$debug" ]; then - echo "$@" >&2 + printf "%s\n" "$*" >&2 fi } say() { if [ -z "$quiet" ]; then - echo "$@" >&2 + printf "%s\n" "$*" >&2 + fi +} + +progress() +{ + if [ -z "$quiet" ]; then + printf "%s\r" "$*" >&2 fi } @@ -297,7 +305,7 @@ copy_commit() # We're going to set some environment vars here, so # do it in a subshell to get rid of them safely later debug copy_commit "{$1}" "{$2}" "{$3}" - git log -1 --pretty=format:'%an%n%ae%n%ad%n%cn%n%ce%n%cd%n%B' "$1" | + git log -1 --pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" | ( read GIT_AUTHOR_NAME read GIT_AUTHOR_EMAIL @@ -311,7 +319,7 @@ copy_commit() GIT_COMMITTER_NAME \ GIT_COMMITTER_EMAIL \ GIT_COMMITTER_DATE - (echo -n "$annotate"; cat ) | + (printf "%s" "$annotate"; cat ) | git commit-tree "$2" $3 # reads the rest of stdin ) || die "Can't copy commit $1" } @@ -489,6 +497,12 @@ ensure_clean() fi } +ensure_valid_ref_format() +{ + git check-ref-format "refs/heads/$1" || + die "'$1' does not look like a ref" +} + cmd_add() { if [ -e "$dir" ]; then @@ -508,8 +522,7 @@ cmd_add() # specified directory. Allowing a refspec might be # misleading because we won't do anything with any other # branches fetched via the refspec. - git rev-parse -q --verify "$2^{commit}" >/dev/null || - die "'$2' does not refer to a commit" + ensure_valid_ref_format "$2" "cmd_add_repository" "$@" else @@ -552,8 +565,9 @@ cmd_add_commit() commit=$(add_squashed_msg "$rev" "$dir" | git commit-tree $tree $headp -p "$rev") || exit $? else + revp=$(peel_committish "$rev") && commit=$(add_msg "$dir" "$headrev" "$rev" | - git commit-tree $tree $headp -p "$rev") || exit $? + git commit-tree $tree $headp -p "$revp") || exit $? fi git reset "$commit" || exit $? @@ -592,7 +606,7 @@ cmd_split() eval "$grl" | while read rev parents; do revcount=$(($revcount + 1)) - say -n "$revcount/$revmax ($createcount)
" + progress "$revcount/$revmax ($createcount)" debug "Processing commit: $rev" exists=$(cache_get $rev) if [ -n "$exists" ]; then @@ -699,7 +713,11 @@ cmd_merge() cmd_pull() { + if [ $# -ne 2 ]; then + die "You must provide <repository> <ref>" + fi ensure_clean + ensure_valid_ref_format "$2" git fetch "$@" || exit $? revs=FETCH_HEAD set -- $revs @@ -709,8 +727,9 @@ cmd_pull() cmd_push() { if [ $# -ne 2 ]; then - die "You must provide <repository> <refspec>" + die "You must provide <repository> <ref>" fi + ensure_valid_ref_format "$2" if [ -e "$dir" ]; then repository=$1 refspec=$2 diff --git a/contrib/subtree/git-subtree.txt b/contrib/subtree/git-subtree.txt index 7ba853eeda..60d76cdddf 100644 --- a/contrib/subtree/git-subtree.txt +++ b/contrib/subtree/git-subtree.txt @@ -9,10 +9,10 @@ git-subtree - Merge subtrees together and split repository into subtrees SYNOPSIS -------- [verse] -'git subtree' add -P <prefix> <refspec> -'git subtree' add -P <prefix> <repository> <refspec> -'git subtree' pull -P <prefix> <repository> <refspec...> -'git subtree' push -P <prefix> <repository> <refspec...> +'git subtree' add -P <prefix> <commit> +'git subtree' add -P <prefix> <repository> <ref> +'git subtree' pull -P <prefix> <repository> <ref> +'git subtree' push -P <prefix> <repository> <ref> 'git subtree' merge -P <prefix> <commit> 'git subtree' split -P <prefix> [OPTIONS] [<commit>] @@ -68,7 +68,7 @@ COMMANDS -------- add:: Create the <prefix> subtree by importing its contents - from the given <refspec> or <repository> and remote <refspec>. + from the given <commit> or <repository> and remote <ref>. A new commit is created automatically, joining the imported project's history with your own. With '--squash', imports only a single commit from the subproject, rather than its @@ -81,22 +81,21 @@ merge:: changes into the latest <commit>. With '--squash', creates only one commit that contains all the changes, rather than merging in the entire history. - - If you use '--squash', the merge direction doesn't - always have to be forward; you can use this command to - go back in time from v2.5 to v2.4, for example. If your - merge introduces a conflict, you can resolve it in the - usual ways. ++ +If you use '--squash', the merge direction doesn't always have to be +forward; you can use this command to go back in time from v2.5 to v2.4, +for example. If your merge introduces a conflict, you can resolve it in +the usual ways. pull:: Exactly like 'merge', but parallels 'git pull' in that - it fetches the given commit from the specified remote + it fetches the given ref from the specified remote repository. push:: Does a 'split' (see below) using the <prefix> supplied and then does a 'git push' to push the result to the - repository and refspec. This can be used to push your + repository and ref. This can be used to push your subtree to different branches of the remote repository. split:: @@ -107,21 +106,19 @@ split:: contents of <prefix> at the root of the project instead of in a subdirectory. Thus, the newly created history is suitable for export as a separate git repository. - - After splitting successfully, a single commit id is - printed to stdout. This corresponds to the HEAD of the - newly created tree, which you can manipulate however you - want. - - Repeated splits of exactly the same history are - guaranteed to be identical (ie. to produce the same - commit ids). Because of this, if you add new commits - and then re-split, the new commits will be attached as - commits on top of the history you generated last time, - so 'git merge' and friends will work as expected. - - Note that if you use '--squash' when you merge, you - should usually not just '--rejoin' when you split. ++ +After splitting successfully, a single commit id is printed to stdout. +This corresponds to the HEAD of the newly created tree, which you can +manipulate however you want. ++ +Repeated splits of exactly the same history are guaranteed to be +identical (i.e. to produce the same commit ids). Because of this, if +you add new commits and then re-split, the new commits will be attached +as commits on top of the history you generated last time, so 'git merge' +and friends will work as expected. ++ +Note that if you use '--squash' when you merge, you should usually not +just '--rejoin' when you split. OPTIONS @@ -149,111 +146,98 @@ OPTIONS OPTIONS FOR add, merge, push, pull ---------------------------------- --squash:: - This option is only valid for add, merge, push and pull + This option is only valid for add, merge, and pull commands. - - Instead of merging the entire history from the subtree - project, produce only a single commit that contains all - the differences you want to merge, and then merge that - new commit into your project. - - Using this option helps to reduce log clutter. People - rarely want to see every change that happened between - v1.0 and v1.1 of the library they're using, since none of the - interim versions were ever included in their application. - - Using '--squash' also helps avoid problems when the same - subproject is included multiple times in the same - project, or is removed and then re-added. In such a - case, it doesn't make sense to combine the histories - anyway, since it's unclear which part of the history - belongs to which subtree. - - Furthermore, with '--squash', you can switch back and - forth between different versions of a subtree, rather - than strictly forward. 'git subtree merge --squash' - always adjusts the subtree to match the exactly - specified commit, even if getting to that commit would - require undoing some changes that were added earlier. - - Whether or not you use '--squash', changes made in your - local repository remain intact and can be later split - and send upstream to the subproject. ++ +Instead of merging the entire history from the subtree project, produce +only a single commit that contains all the differences you want to +merge, and then merge that new commit into your project. ++ +Using this option helps to reduce log clutter. People rarely want to see +every change that happened between v1.0 and v1.1 of the library they're +using, since none of the interim versions were ever included in their +application. ++ +Using '--squash' also helps avoid problems when the same subproject is +included multiple times in the same project, or is removed and then +re-added. In such a case, it doesn't make sense to combine the +histories anyway, since it's unclear which part of the history belongs +to which subtree. ++ +Furthermore, with '--squash', you can switch back and forth between +different versions of a subtree, rather than strictly forward. 'git +subtree merge --squash' always adjusts the subtree to match the exactly +specified commit, even if getting to that commit would require undoing +some changes that were added earlier. ++ +Whether or not you use '--squash', changes made in your local repository +remain intact and can be later split and send upstream to the +subproject. OPTIONS FOR split ----------------- --annotate=<annotation>:: This option is only valid for the split command. - - When generating synthetic history, add <annotation> as a - prefix to each commit message. Since we're creating new - commits with the same commit message, but possibly - different content, from the original commits, this can help - to differentiate them and avoid confusion. - - Whenever you split, you need to use the same - <annotation>, or else you don't have a guarantee that - the new re-created history will be identical to the old - one. That will prevent merging from working correctly. - git subtree tries to make it work anyway, particularly - if you use --rejoin, but it may not always be effective. ++ +When generating synthetic history, add <annotation> as a prefix to each +commit message. Since we're creating new commits with the same commit +message, but possibly different content, from the original commits, this +can help to differentiate them and avoid confusion. ++ +Whenever you split, you need to use the same <annotation>, or else you +don't have a guarantee that the new re-created history will be identical +to the old one. That will prevent merging from working correctly. git +subtree tries to make it work anyway, particularly if you use --rejoin, +but it may not always be effective. -b <branch>:: --branch=<branch>:: This option is only valid for the split command. - - After generating the synthetic history, create a new - branch called <branch> that contains the new history. - This is suitable for immediate pushing upstream. - <branch> must not already exist. ++ +After generating the synthetic history, create a new branch called +<branch> that contains the new history. This is suitable for immediate +pushing upstream. <branch> must not already exist. --ignore-joins:: This option is only valid for the split command. - - If you use '--rejoin', git subtree attempts to optimize - its history reconstruction to generate only the new - commits since the last '--rejoin'. '--ignore-join' - disables this behaviour, forcing it to regenerate the - entire history. In a large project, this can take a - long time. ++ +If you use '--rejoin', git subtree attempts to optimize its history +reconstruction to generate only the new commits since the last +'--rejoin'. '--ignore-join' disables this behaviour, forcing it to +regenerate the entire history. In a large project, this can take a long +time. --onto=<onto>:: This option is only valid for the split command. - - If your subtree was originally imported using something - other than git subtree, its history may not match what - git subtree is expecting. In that case, you can specify - the commit id <onto> that corresponds to the first - revision of the subproject's history that was imported - into your project, and git subtree will attempt to build - its history from there. - - If you used 'git subtree add', you should never need - this option. ++ +If your subtree was originally imported using something other than git +subtree, its history may not match what git subtree is expecting. In +that case, you can specify the commit id <onto> that corresponds to the +first revision of the subproject's history that was imported into your +project, and git subtree will attempt to build its history from there. ++ +If you used 'git subtree add', you should never need this option. --rejoin:: This option is only valid for the split command. - - After splitting, merge the newly created synthetic - history back into your main project. That way, future - splits can search only the part of history that has - been added since the most recent --rejoin. - - If your split commits end up merged into the upstream - subproject, and then you want to get the latest upstream - version, this will allow git's merge algorithm to more - intelligently avoid conflicts (since it knows these - synthetic commits are already part of the upstream - repository). - - Unfortunately, using this option results in 'git log' - showing an extra copy of every new commit that was - created (the original, and the synthetic one). - - If you do all your merges with '--squash', don't use - '--rejoin' when you split, because you don't want the - subproject's history to be part of your project anyway. ++ +After splitting, merge the newly created synthetic history back into +your main project. That way, future splits can search only the part of +history that has been added since the most recent --rejoin. ++ +If your split commits end up merged into the upstream subproject, and +then you want to get the latest upstream version, this will allow git's +merge algorithm to more intelligently avoid conflicts (since it knows +these synthetic commits are already part of the upstream repository). ++ +Unfortunately, using this option results in 'git log' showing an extra +copy of every new commit that was created (the original, and the +synthetic one). ++ +If you do all your merges with '--squash', don't use '--rejoin' when you +split, because you don't want the subproject's history to be part of +your project anyway. EXAMPLE 1. Add command @@ -270,7 +254,7 @@ git-extensions repository in ~/git-extensions/: name You can omit the --squash flag, but doing so will increase the number -of commits that are incldued in your local repository. +of commits that are included in your local repository. We now have a ~/git-extensions/git-subtree directory containing code from the master branch of git://github.com/apenwarr/git-subtree.git diff --git a/contrib/subtree/t/t7900-subtree.sh b/contrib/subtree/t/t7900-subtree.sh index b0f8536fca..90519823be 100755 --- a/contrib/subtree/t/t7900-subtree.sh +++ b/contrib/subtree/t/t7900-subtree.sh @@ -8,7 +8,8 @@ This test verifies the basic operation of the merge, pull, add and split subcommands of git subtree. ' -export TEST_DIRECTORY=$(pwd)/../../../t +TEST_DIRECTORY=$(pwd)/../../../t +export TEST_DIRECTORY . ../../../t/test-lib.sh @@ -61,151 +62,156 @@ last_commit_message() } test_expect_success 'init subproj' ' - test_create_repo subproj + test_create_repo subproj ' # To the subproject! cd subproj test_expect_success 'add sub1' ' - create sub1 && - git commit -m "sub1" && - git branch sub1 && - git branch -m master subproj + create sub1 && + git commit -m "sub1" && + git branch sub1 && + git branch -m master subproj ' # Save this hash for testing later. -subdir_hash=`git rev-parse HEAD` +subdir_hash=$(git rev-parse HEAD) test_expect_success 'add sub2' ' - create sub2 && - git commit -m "sub2" && - git branch sub2 + create sub2 && + git commit -m "sub2" && + git branch sub2 ' test_expect_success 'add sub3' ' - create sub3 && - git commit -m "sub3" && - git branch sub3 + create sub3 && + git commit -m "sub3" && + git branch sub3 ' # Back to mainline cd .. +test_expect_success 'enable log.date=relative to catch errors' ' + git config log.date relative +' + test_expect_success 'add main4' ' - create main4 && - git commit -m "main4" && - git branch -m master mainline && - git branch subdir + create main4 && + git commit -m "main4" && + git branch -m master mainline && + git branch subdir ' test_expect_success 'fetch subproj history' ' - git fetch ./subproj sub1 && - git branch sub1 FETCH_HEAD + git fetch ./subproj sub1 && + git branch sub1 FETCH_HEAD ' test_expect_success 'no subtree exists in main tree' ' - test_must_fail git subtree merge --prefix=subdir sub1 + test_must_fail git subtree merge --prefix=subdir sub1 ' test_expect_success 'no pull from non-existant subtree' ' - test_must_fail git subtree pull --prefix=subdir ./subproj sub1 + test_must_fail git subtree pull --prefix=subdir ./subproj sub1 ' test_expect_success 'check if --message works for add' ' - git subtree add --prefix=subdir --message="Added subproject" sub1 && - check_equal ''"$(last_commit_message)"'' "Added subproject" && - undo + git subtree add --prefix=subdir --message="Added subproject" sub1 && + check_equal ''"$(last_commit_message)"'' "Added subproject" && + undo ' test_expect_success 'check if --message works as -m and --prefix as -P' ' - git subtree add -P subdir -m "Added subproject using git subtree" sub1 && - check_equal ''"$(last_commit_message)"'' "Added subproject using git subtree" && - undo + git subtree add -P subdir -m "Added subproject using git subtree" sub1 && + check_equal ''"$(last_commit_message)"'' "Added subproject using git subtree" && + undo ' test_expect_success 'check if --message works with squash too' ' - git subtree add -P subdir -m "Added subproject with squash" --squash sub1 && - check_equal ''"$(last_commit_message)"'' "Added subproject with squash" && - undo + git subtree add -P subdir -m "Added subproject with squash" --squash sub1 && + check_equal ''"$(last_commit_message)"'' "Added subproject with squash" && + undo ' test_expect_success 'add subproj to mainline' ' - git subtree add --prefix=subdir/ FETCH_HEAD && - check_equal ''"$(last_commit_message)"'' "Add '"'subdir/'"' from commit '"'"'''"$(git rev-parse sub1)"'''"'"'" + git subtree add --prefix=subdir/ FETCH_HEAD && + check_equal ''"$(last_commit_message)"'' "Add '"'subdir/'"' from commit '"'"'''"$(git rev-parse sub1)"'''"'"'" ' # this shouldn't actually do anything, since FETCH_HEAD is already a parent test_expect_success 'merge fetched subproj' ' - git merge -m "merge -s -ours" -s ours FETCH_HEAD + git merge -m "merge -s -ours" -s ours FETCH_HEAD ' test_expect_success 'add main-sub5' ' - create subdir/main-sub5 && - git commit -m "main-sub5" + create subdir/main-sub5 && + git commit -m "main-sub5" ' test_expect_success 'add main6' ' - create main6 && - git commit -m "main6 boring" + create main6 && + git commit -m "main6 boring" ' test_expect_success 'add main-sub7' ' - create subdir/main-sub7 && - git commit -m "main-sub7" + create subdir/main-sub7 && + git commit -m "main-sub7" ' test_expect_success 'fetch new subproj history' ' - git fetch ./subproj sub2 && - git branch sub2 FETCH_HEAD + git fetch ./subproj sub2 && + git branch sub2 FETCH_HEAD ' test_expect_success 'check if --message works for merge' ' - git subtree merge --prefix=subdir -m "Merged changes from subproject" sub2 && - check_equal ''"$(last_commit_message)"'' "Merged changes from subproject" && - undo + git subtree merge --prefix=subdir -m "Merged changes from subproject" sub2 && + check_equal ''"$(last_commit_message)"'' "Merged changes from subproject" && + undo ' test_expect_success 'check if --message for merge works with squash too' ' - git subtree merge --prefix subdir -m "Merged changes from subproject using squash" --squash sub2 && - check_equal ''"$(last_commit_message)"'' "Merged changes from subproject using squash" && - undo + git subtree merge --prefix subdir -m "Merged changes from subproject using squash" --squash sub2 && + check_equal ''"$(last_commit_message)"'' "Merged changes from subproject using squash" && + undo ' test_expect_success 'merge new subproj history into subdir' ' - git subtree merge --prefix=subdir FETCH_HEAD && - git branch pre-split && - check_equal ''"$(last_commit_message)"'' "Merge commit '"'"'"$(git rev-parse sub2)"'"'"' into mainline" + git subtree merge --prefix=subdir FETCH_HEAD && + git branch pre-split && + check_equal ''"$(last_commit_message)"'' "Merge commit '"'"'"$(git rev-parse sub2)"'"'"' into mainline" && + undo ' test_expect_success 'Check that prefix argument is required for split' ' - echo "You must provide the --prefix option." > expected && - test_must_fail git subtree split > actual 2>&1 && - test_debug "echo -n expected: " && - test_debug "cat expected" && - test_debug "echo -n actual: " && - test_debug "cat actual" && - test_cmp expected actual && - rm -f expected actual + echo "You must provide the --prefix option." > expected && + test_must_fail git subtree split > actual 2>&1 && + test_debug "printf '"'"'expected: '"'"'" && + test_debug "cat expected" && + test_debug "printf '"'"'actual: '"'"'" && + test_debug "cat actual" && + test_cmp expected actual && + rm -f expected actual ' test_expect_success 'Check that the <prefix> exists for a split' ' - echo "'"'"'non-existent-directory'"'"'" does not exist\; use "'"'"'git subtree add'"'"'" > expected && - test_must_fail git subtree split --prefix=non-existent-directory > actual 2>&1 && - test_debug "echo -n expected: " && - test_debug "cat expected" && - test_debug "echo -n actual: " && - test_debug "cat actual" && - test_cmp expected actual -# rm -f expected actual + echo "'"'"'non-existent-directory'"'"'" does not exist\; use "'"'"'git subtree add'"'"'" > expected && + test_must_fail git subtree split --prefix=non-existent-directory > actual 2>&1 && + test_debug "printf '"'"'expected: '"'"'" && + test_debug "cat expected" && + test_debug "printf '"'"'actual: '"'"'" && + test_debug "cat actual" && + test_cmp expected actual +# rm -f expected actual ' test_expect_success 'check if --message works for split+rejoin' ' - spl1=''"$(git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --message "Split & rejoin" --rejoin)"'' && - git branch spl1 "$spl1" && - check_equal ''"$(last_commit_message)"'' "Split & rejoin" && - undo + spl1=''"$(git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --message "Split & rejoin" --rejoin)"'' && + git branch spl1 "$spl1" && + check_equal ''"$(last_commit_message)"'' "Split & rejoin" && + undo ' test_expect_success 'check split with --branch' ' @@ -217,79 +223,76 @@ test_expect_success 'check split with --branch' ' test_expect_success 'check hash of split' ' spl1=$(git subtree split --prefix subdir) && - undo && git subtree split --prefix subdir --branch splitbr1test && - check_equal ''"$(git rev-parse splitbr1test)"'' "$spl1" - git checkout splitbr1test && - new_hash=$(git rev-parse HEAD~2) && - git checkout mainline && + check_equal ''"$(git rev-parse splitbr1test)"'' "$spl1" && + new_hash=$(git rev-parse splitbr1test~2) && check_equal ''"$new_hash"'' "$subdir_hash" ' test_expect_success 'check split with --branch for an existing branch' ' - spl1=''"$(git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --message "Split & rejoin" --rejoin)"'' && - undo && - git branch splitbr2 sub1 && - git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --branch splitbr2 && - check_equal ''"$(git rev-parse splitbr2)"'' "$spl1" + spl1=''"$(git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --message "Split & rejoin" --rejoin)"'' && + undo && + git branch splitbr2 sub1 && + git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --branch splitbr2 && + check_equal ''"$(git rev-parse splitbr2)"'' "$spl1" ' test_expect_success 'check split with --branch for an incompatible branch' ' - test_must_fail git subtree split --prefix subdir --onto FETCH_HEAD --branch subdir + test_must_fail git subtree split --prefix subdir --onto FETCH_HEAD --branch subdir ' test_expect_success 'check split+rejoin' ' - spl1=''"$(git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --message "Split & rejoin" --rejoin)"'' && - undo && - git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --rejoin && - check_equal ''"$(last_commit_message)"'' "Split '"'"'subdir/'"'"' into commit '"'"'"$spl1"'"'"'" + spl1=''"$(git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --message "Split & rejoin" --rejoin)"'' && + undo && + git subtree split --annotate='"'*'"' --prefix subdir --onto FETCH_HEAD --rejoin && + check_equal ''"$(last_commit_message)"'' "Split '"'"'subdir/'"'"' into commit '"'"'"$spl1"'"'"'" ' test_expect_success 'add main-sub8' ' - create subdir/main-sub8 && - git commit -m "main-sub8" + create subdir/main-sub8 && + git commit -m "main-sub8" ' # To the subproject! cd ./subproj test_expect_success 'merge split into subproj' ' - git fetch .. spl1 && - git branch spl1 FETCH_HEAD && - git merge FETCH_HEAD + git fetch .. spl1 && + git branch spl1 FETCH_HEAD && + git merge FETCH_HEAD ' test_expect_success 'add sub9' ' - create sub9 && - git commit -m "sub9" + create sub9 && + git commit -m "sub9" ' # Back to mainline cd .. test_expect_success 'split for sub8' ' - split2=''"$(git subtree split --annotate='"'*'"' --prefix subdir/ --rejoin)"'' - git branch split2 "$split2" + split2=''"$(git subtree split --annotate='"'*'"' --prefix subdir/ --rejoin)"'' && + git branch split2 "$split2" ' test_expect_success 'add main-sub10' ' - create subdir/main-sub10 && - git commit -m "main-sub10" + create subdir/main-sub10 && + git commit -m "main-sub10" ' test_expect_success 'split for sub10' ' - spl3=''"$(git subtree split --annotate='"'*'"' --prefix subdir --rejoin)"'' && - git branch spl3 "$spl3" + spl3=''"$(git subtree split --annotate='"'*'"' --prefix subdir --rejoin)"'' && + git branch spl3 "$spl3" ' # To the subproject! cd ./subproj test_expect_success 'merge split into subproj' ' - git fetch .. spl3 && - git branch spl3 FETCH_HEAD && - git merge FETCH_HEAD && - git branch subproj-merge-spl3 + git fetch .. spl3 && + git branch spl3 FETCH_HEAD && + git merge FETCH_HEAD && + git branch subproj-merge-spl3 ' chkm="main4 main6" @@ -299,44 +302,44 @@ chks="sub1 sub2 sub3 sub9" chks_sub=$(echo $chks | multiline | sed 's,^,subdir/,' | fixnl) test_expect_success 'make sure exactly the right set of files ends up in the subproj' ' - subfiles=''"$(git ls-files | fixnl)"'' && - check_equal "$subfiles" "$chkms $chks" + subfiles=''"$(git ls-files | fixnl)"'' && + check_equal "$subfiles" "$chkms $chks" ' test_expect_success 'make sure the subproj history *only* contains commits that affect the subdir' ' - allchanges=''"$(git log --name-only --pretty=format:'"''"' | sort | fixnl)"'' && - check_equal "$allchanges" "$chkms $chks" + allchanges=''"$(git log --name-only --pretty=format:'"''"' | sort | fixnl)"'' && + check_equal "$allchanges" "$chkms $chks" ' # Back to mainline cd .. test_expect_success 'pull from subproj' ' - git fetch ./subproj subproj-merge-spl3 && - git branch subproj-merge-spl3 FETCH_HEAD && - git subtree pull --prefix=subdir ./subproj subproj-merge-spl3 + git fetch ./subproj subproj-merge-spl3 && + git branch subproj-merge-spl3 FETCH_HEAD && + git subtree pull --prefix=subdir ./subproj subproj-merge-spl3 ' test_expect_success 'make sure exactly the right set of files ends up in the mainline' ' - mainfiles=''"$(git ls-files | fixnl)"'' && - check_equal "$mainfiles" "$chkm $chkms_sub $chks_sub" + mainfiles=''"$(git ls-files | fixnl)"'' && + check_equal "$mainfiles" "$chkm $chkms_sub $chks_sub" ' test_expect_success 'make sure each filename changed exactly once in the entire history' ' - # main-sub?? and /subdir/main-sub?? both change, because those are the - # changes that were split into their own history. And subdir/sub?? never - # change, since they were *only* changed in the subtree branch. - allchanges=''"$(git log --name-only --pretty=format:'"''"' | sort | fixnl)"'' && - check_equal "$allchanges" ''"$(echo $chkms $chkm $chks $chkms_sub | multiline | sort | fixnl)"'' + # main-sub?? and /subdir/main-sub?? both change, because those are the + # changes that were split into their own history. And subdir/sub?? never + # change, since they were *only* changed in the subtree branch. + allchanges=''"$(git log --name-only --pretty=format:'"''"' | sort | fixnl)"'' && + check_equal "$allchanges" ''"$(echo $chkms $chkm $chks $chkms_sub | multiline | sort | fixnl)"'' ' test_expect_success 'make sure the --rejoin commits never make it into subproj' ' - check_equal ''"$(git log --pretty=format:'"'%s'"' HEAD^2 | grep -i split)"'' "" + check_equal ''"$(git log --pretty=format:'"'%s'"' HEAD^2 | grep -i split)"'' "" ' test_expect_success 'make sure no "git subtree" tagged commits make it into subproj' ' - # They are meaningless to subproj since one side of the merge refers to the mainline - check_equal ''"$(git log --pretty=format:'"'%s%n%b'"' HEAD^2 | grep "git-subtree.*:")"'' "" + # They are meaningless to subproj since one side of the merge refers to the mainline + check_equal ''"$(git log --pretty=format:'"'%s%n%b'"' HEAD^2 | grep "git-subtree.*:")"'' "" ' # prepare second pair of repositories @@ -344,27 +347,27 @@ mkdir test2 cd test2 test_expect_success 'init main' ' - test_create_repo main + test_create_repo main ' cd main test_expect_success 'add main1' ' - create main1 && - git commit -m "main1" + create main1 && + git commit -m "main1" ' cd .. test_expect_success 'init sub' ' - test_create_repo sub + test_create_repo sub ' cd sub test_expect_success 'add sub2' ' - create sub2 && - git commit -m "sub2" + create sub2 && + git commit -m "sub2" ' cd ../main @@ -372,33 +375,33 @@ cd ../main # check if split can find proper base without --onto test_expect_success 'add sub as subdir in main' ' - git fetch ../sub master && - git branch sub2 FETCH_HEAD && - git subtree add --prefix subdir sub2 + git fetch ../sub master && + git branch sub2 FETCH_HEAD && + git subtree add --prefix subdir sub2 ' cd ../sub test_expect_success 'add sub3' ' - create sub3 && - git commit -m "sub3" + create sub3 && + git commit -m "sub3" ' cd ../main test_expect_success 'merge from sub' ' - git fetch ../sub master && - git branch sub3 FETCH_HEAD && - git subtree merge --prefix subdir sub3 + git fetch ../sub master && + git branch sub3 FETCH_HEAD && + git subtree merge --prefix subdir sub3 ' test_expect_success 'add main-sub4' ' - create subdir/main-sub4 && - git commit -m "main-sub4" + create subdir/main-sub4 && + git commit -m "main-sub4" ' test_expect_success 'split for main-sub4 without --onto' ' - git subtree split --prefix subdir --branch mainsub4 + git subtree split --prefix subdir --branch mainsub4 ' # at this point, the new commit parent should be sub3 if it is not, @@ -407,21 +410,21 @@ test_expect_success 'split for main-sub4 without --onto' ' # itself) test_expect_success 'check that the commit parent is sub3' ' - check_equal ''"$(git log --pretty=format:%P -1 mainsub4)"'' ''"$(git rev-parse sub3)"'' + check_equal ''"$(git log --pretty=format:%P -1 mainsub4)"'' ''"$(git rev-parse sub3)"'' ' test_expect_success 'add main-sub5' ' - mkdir subdir2 && - create subdir2/main-sub5 && - git commit -m "main-sub5" + mkdir subdir2 && + create subdir2/main-sub5 && + git commit -m "main-sub5" ' test_expect_success 'split for main-sub5 without --onto' ' - # also test that we still can split out an entirely new subtree - # if the parent of the first commit in the tree is not empty, + # also test that we still can split out an entirely new subtree + # if the parent of the first commit in the tree is not empty, # then the new subtree has accidentally been attached to something - git subtree split --prefix subdir2 --branch mainsub5 && - check_equal ''"$(git log --pretty=format:%P -1 mainsub5)"'' "" + git subtree split --prefix subdir2 --branch mainsub5 && + check_equal ''"$(git log --pretty=format:%P -1 mainsub5)"'' "" ' # make sure no patch changes more than one file. The original set of commits @@ -449,20 +452,20 @@ joincommits() } test_expect_success 'verify one file change per commit' ' - x= && - list=''"$(git log --pretty=format:'"'commit: %H'"' | joincommits)"'' && -# test_debug "echo HERE" && -# test_debug "echo ''"$list"''" && - (git log --pretty=format:'"'commit: %H'"' | joincommits | - ( while read commit a b; do - test_debug "echo Verifying commit "''"$commit"'' - test_debug "echo a: "''"$a"'' - test_debug "echo b: "''"$b"'' - check_equal "$b" "" - x=1 - done - check_equal "$x" 1 - )) + x= && + list=''"$(git log --pretty=format:'"'commit: %H'"' | joincommits)"'' && +# test_debug "echo HERE" && +# test_debug "echo ''"$list"''" && + (git log --pretty=format:'"'commit: %H'"' | joincommits | + ( while read commit a b; do + test_debug "echo Verifying commit "''"$commit"'' + test_debug "echo a: "''"$a"'' + test_debug "echo b: "''"$b"'' + check_equal "$b" "" + x=1 + done + check_equal "$x" 1 + )) ' test_done diff --git a/contrib/svn-fe/Makefile b/contrib/svn-fe/Makefile index 360d8da417..e8651aaf4b 100644 --- a/contrib/svn-fe/Makefile +++ b/contrib/svn-fe/Makefile @@ -1,18 +1,58 @@ all:: svn-fe$X -CC = gcc +CC = cc RM = rm -f MV = mv CFLAGS = -g -O2 -Wall LDFLAGS = -ALL_CFLAGS = $(CFLAGS) -ALL_LDFLAGS = $(LDFLAGS) -EXTLIBS = +EXTLIBS = -lz + +include ../../config.mak.uname +-include ../../config.mak.autogen +-include ../../config.mak + +ifeq ($(uname_S),Darwin) + ifndef NO_FINK + ifeq ($(shell test -d /sw/lib && echo y),y) + CFLAGS += -I/sw/include + LDFLAGS += -L/sw/lib + endif + endif + ifndef NO_DARWIN_PORTS + ifeq ($(shell test -d /opt/local/lib && echo y),y) + CFLAGS += -I/opt/local/include + LDFLAGS += -L/opt/local/lib + endif + endif +endif + +ifndef NO_OPENSSL + EXTLIBS += -lssl + ifdef NEEDS_CRYPTO_WITH_SSL + EXTLIBS += -lcrypto + endif +endif + +ifndef NO_PTHREADS + CFLAGS += $(PTHREADS_CFLAGS) + EXTLIBS += $(PTHREAD_LIBS) +endif + +ifdef HAVE_CLOCK_GETTIME + CFLAGS += -DHAVE_CLOCK_GETTIME + EXTLIBS += -lrt +endif + +ifdef NEEDS_LIBICONV + EXTLIBS += -liconv +endif GIT_LIB = ../../libgit.a VCSSVN_LIB = ../../vcs-svn/lib.a -LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(EXTLIBS) +XDIFF_LIB = ../../xdiff/lib.a + +LIBS = $(VCSSVN_LIB) $(GIT_LIB) $(XDIFF_LIB) QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir QUIET_SUBDIR1 = @@ -33,12 +73,11 @@ ifndef V endif endif -svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(GIT_LIB) - $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ svn-fe.o \ - $(ALL_LDFLAGS) $(LIBS) +svn-fe$X: svn-fe.o $(VCSSVN_LIB) $(XDIFF_LIB) $(GIT_LIB) + $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $(EXTLIBS) -o $@ svn-fe.o $(LIBS) svn-fe.o: svn-fe.c ../../vcs-svn/svndump.h - $(QUIET_CC)$(CC) -I../../vcs-svn -o $*.o -c $(ALL_CFLAGS) $< + $(QUIET_CC)$(CC) $(CFLAGS) -I../../vcs-svn -o $*.o -c $< svn-fe.html: svn-fe.txt $(QUIET_SUBDIR0)../../Documentation $(QUIET_SUBDIR1) \ @@ -54,6 +93,9 @@ svn-fe.1: svn-fe.txt ../../vcs-svn/lib.a: FORCE $(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) vcs-svn/lib.a +../../xdiff/lib.a: FORCE + $(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) xdiff/lib.a + ../../libgit.a: FORCE $(QUIET_SUBDIR0)../.. $(QUIET_SUBDIR1) libgit.a diff --git a/contrib/svn-fe/svn-fe.txt b/contrib/svn-fe/svn-fe.txt index 1128ab2ce4..a3425f4770 100644 --- a/contrib/svn-fe/svn-fe.txt +++ b/contrib/svn-fe/svn-fe.txt @@ -40,8 +40,8 @@ manual page. NOTES ----- Subversion dumps do not record a separate author and committer for -each revision, nor a separate display name and email address for -each author. Like git-svn(1), 'svn-fe' will use the name +each revision, nor do they record a separate display name and email +address for each author. Like git-svn(1), 'svn-fe' will use the name --------- user <user@UUID> diff --git a/contrib/svn-fe/svnrdump_sim.py b/contrib/svn-fe/svnrdump_sim.py index 4e78a1c3cd..11ac6f6927 100755 --- a/contrib/svn-fe/svnrdump_sim.py +++ b/contrib/svn-fe/svnrdump_sim.py @@ -5,53 +5,64 @@ of the specified revision range. To simulate incremental imports the environment variable SVNRMAX can be set to the highest revision that should be available. """ -import sys, os +import sys +import os if sys.hexversion < 0x02040000: - # The limiter is the ValueError() calls. This may be too conservative - sys.stderr.write("svnrdump-sim.py: requires Python 2.4 or later.\n") - sys.exit(1) + # The limiter is the ValueError() calls. This may be too conservative + sys.stderr.write("svnrdump-sim.py: requires Python 2.4 or later.\n") + sys.exit(1) + def getrevlimit(): - var = 'SVNRMAX' - if var in os.environ: - return os.environ[var] - return None + var = 'SVNRMAX' + if var in os.environ: + return os.environ[var] + return None + def writedump(url, lower, upper): - if url.startswith('sim://'): - filename = url[6:] - if filename[-1] == '/': filename = filename[:-1] #remove terminating slash - else: - raise ValueError('sim:// url required') - f = open(filename, 'r'); - state = 'header' - wroterev = False - while(True): - l = f.readline() - if l == '': break - if state == 'header' and l.startswith('Revision-number: '): - state = 'prefix' - if state == 'prefix' and l == 'Revision-number: %s\n' % lower: - state = 'selection' - if not upper == 'HEAD' and state == 'selection' and l == 'Revision-number: %s\n' % upper: - break; + if url.startswith('sim://'): + filename = url[6:] + if filename[-1] == '/': + filename = filename[:-1] # remove terminating slash + else: + raise ValueError('sim:// url required') + f = open(filename, 'r') + state = 'header' + wroterev = False + while(True): + l = f.readline() + if l == '': + break + if state == 'header' and l.startswith('Revision-number: '): + state = 'prefix' + if state == 'prefix' and l == 'Revision-number: %s\n' % lower: + state = 'selection' + if not upper == 'HEAD' and state == 'selection' and \ + l == 'Revision-number: %s\n' % upper: + break - if state == 'header' or state == 'selection': - if state == 'selection': wroterev = True - sys.stdout.write(l) - return wroterev + if state == 'header' or state == 'selection': + if state == 'selection': + wroterev = True + sys.stdout.write(l) + return wroterev if __name__ == "__main__": - if not (len(sys.argv) in (3, 4, 5)): - print("usage: %s dump URL -rLOWER:UPPER") - sys.exit(1) - if not sys.argv[1] == 'dump': raise NotImplementedError('only "dump" is suppported.') - url = sys.argv[2] - r = ('0', 'HEAD') - if len(sys.argv) == 4 and sys.argv[3][0:2] == '-r': - r = sys.argv[3][2:].lstrip().split(':') - if not getrevlimit() is None: r[1] = getrevlimit() - if writedump(url, r[0], r[1]): ret = 0 - else: ret = 1 - sys.exit(ret) + if not (len(sys.argv) in (3, 4, 5)): + print("usage: %s dump URL -rLOWER:UPPER") + sys.exit(1) + if not sys.argv[1] == 'dump': + raise NotImplementedError('only "dump" is suppported.') + url = sys.argv[2] + r = ('0', 'HEAD') + if len(sys.argv) == 4 and sys.argv[3][0:2] == '-r': + r = sys.argv[3][2:].lstrip().split(':') + if not getrevlimit() is None: + r[1] = getrevlimit() + if writedump(url, r[0], r[1]): + ret = 0 + else: + ret = 1 + sys.exit(ret) diff --git a/contrib/thunderbird-patch-inline/appp.sh b/contrib/thunderbird-patch-inline/appp.sh index 5eb4a51643..8dc73ece15 100755 --- a/contrib/thunderbird-patch-inline/appp.sh +++ b/contrib/thunderbird-patch-inline/appp.sh @@ -10,7 +10,7 @@ CONFFILE=~/.appprc SEP="-=-=-=-=-=-=-=-=-=# Don't remove this line #=-=-=-=-=-=-=-=-=-" if [ -e "$CONFFILE" ] ; then - LAST_DIR=`grep -m 1 "^LAST_DIR=" "${CONFFILE}"|sed -e 's/^LAST_DIR=//'` + LAST_DIR=$(grep -m 1 "^LAST_DIR=" "${CONFFILE}"|sed -e 's/^LAST_DIR=//') cd "${LAST_DIR}" else cd > /dev/null @@ -25,11 +25,11 @@ fi cd - > /dev/null -SUBJECT=`sed -n -e '/^Subject: /p' "${PATCH}"` -HEADERS=`sed -e '/^'"${SEP}"'$/,$d' $1` -BODY=`sed -e "1,/${SEP}/d" $1` -CMT_MSG=`sed -e '1,/^$/d' -e '/^---$/,$d' "${PATCH}"` -DIFF=`sed -e '1,/^---$/d' "${PATCH}"` +SUBJECT=$(sed -n -e '/^Subject: /p' "${PATCH}") +HEADERS=$(sed -e '/^'"${SEP}"'$/,$d' $1) +BODY=$(sed -e "1,/${SEP}/d" $1) +CMT_MSG=$(sed -e '1,/^$/d' -e '/^---$/,$d' "${PATCH}") +DIFF=$(sed -e '1,/^---$/d' "${PATCH}") CCS=`echo -e "$CMT_MSG\n$HEADERS" | sed -n -e 's/^Cc: \(.*\)$/\1,/gp' \ -e 's/^Signed-off-by: \(.*\)/\1,/gp'` @@ -48,7 +48,7 @@ if [ "x${BODY}x" != "xx" ] ; then fi echo "$DIFF" >> $1 -LAST_DIR=`dirname "${PATCH}"` +LAST_DIR=$(dirname "${PATCH}") grep -v "^LAST_DIR=" "${CONFFILE}" > "${CONFFILE}_" echo "LAST_DIR=${LAST_DIR}" >> "${CONFFILE}_" diff --git a/contrib/vim/README b/contrib/vim/README deleted file mode 100644 index 8f16d06972..0000000000 --- a/contrib/vim/README +++ /dev/null @@ -1,22 +0,0 @@ -Syntax highlighting for git commit messages, config files, etc. is -included with the vim distribution as of vim 7.2, and should work -automatically. - -If you have an older version of vim, you can get the latest syntax -files from the vim project: - - http://ftp.vim.org/pub/vim/runtime/syntax/git.vim - http://ftp.vim.org/pub/vim/runtime/syntax/gitcommit.vim - http://ftp.vim.org/pub/vim/runtime/syntax/gitconfig.vim - http://ftp.vim.org/pub/vim/runtime/syntax/gitrebase.vim - http://ftp.vim.org/pub/vim/runtime/syntax/gitsendemail.vim - -These files are also available via FTP at the same location. - -To install: - - 1. Copy these files to vim's syntax directory $HOME/.vim/syntax - 2. To auto-detect the editing of various git-related filetypes: - - $ curl http://ftp.vim.org/pub/vim/runtime/filetype.vim | - sed -ne '/^" Git$/, /^$/ p' >>$HOME/.vim/filetype.vim diff --git a/contrib/workdir/git-new-workdir b/contrib/workdir/git-new-workdir index 75e8b25817..888c34a521 100755 --- a/contrib/workdir/git-new-workdir +++ b/contrib/workdir/git-new-workdir @@ -10,6 +10,10 @@ die () { exit 128 } +failed () { + die "unable to create new workdir '$new_workdir'!" +} + if test $# -lt 2 || test $# -gt 3 then usage "$0 <repository> <new_workdir> [<branch>]" @@ -35,7 +39,7 @@ esac # don't link to a configured bare repository isbare=$(git --git-dir="$git_dir" config --bool --get core.bare) -if test ztrue = z$isbare +if test ztrue = "z$isbare" then die "\"$git_dir\" has core.bare set to true," \ " remove from \"$git_dir/config\" to use $0" @@ -48,35 +52,54 @@ then "a complete repository." fi -# don't recreate a workdir over an existing repository -if test -e "$new_workdir" +# make sure the links in the workdir have full paths to the original repo +git_dir=$(cd "$git_dir" && pwd) || exit 1 + +# don't recreate a workdir over an existing directory, unless it's empty +if test -d "$new_workdir" then - die "destination directory '$new_workdir' already exists." + if test $(ls -a1 "$new_workdir/." | wc -l) -ne 2 + then + die "destination directory '$new_workdir' is not empty." + fi + cleandir="$new_workdir/.git" +else + cleandir="$new_workdir" fi -# make sure the links use full paths -git_dir=$(cd "$git_dir"; pwd) +mkdir -p "$new_workdir/.git" || failed +cleandir=$(cd "$cleandir" && pwd) || failed -# create the workdir -mkdir -p "$new_workdir/.git" || die "unable to create \"$new_workdir\"!" +cleanup () { + rm -rf "$cleandir" +} +siglist="0 1 2 15" +trap cleanup $siglist # create the links to the original repo. explicitly exclude index, HEAD and # logs/HEAD from the list since they are purely related to the current working # directory, and should not be shared. for x in config refs logs/refs objects info hooks packed-refs remotes rr-cache svn do + # create a containing directory if needed case $x in */*) - mkdir -p "$(dirname "$new_workdir/.git/$x")" + mkdir -p "$new_workdir/.git/${x%/*}" ;; esac - ln -s "$git_dir/$x" "$new_workdir/.git/$x" + + ln -s "$git_dir/$x" "$new_workdir/.git/$x" || failed done -# now setup the workdir -cd "$new_workdir" +# commands below this are run in the context of the new workdir +cd "$new_workdir" || failed + # copy the HEAD from the original repository as a default branch -cp "$git_dir/HEAD" .git/HEAD -# checkout the branch (either the same as HEAD from the original repository, or -# the one that was asked for) +cp "$git_dir/HEAD" .git/HEAD || failed + +# the workdir is set up. if the checkout fails, the user can fix it. +trap - $siglist + +# checkout the branch (either the same as HEAD from the original repository, +# or the one that was asked for) git checkout -f $branch |