summaryrefslogtreecommitdiff
path: root/contrib/mw-to-git/git-remote-mediawiki
AgeCommit message (Collapse)AuthorFilesLines
2012-08-03git-remote-mediawiki: replace TODO-list in comment by appropriate linkLibravatar Matthieu Moy1-14/+1
My account on Github is now used as wiki and issue tracking. This will be more flexible than in-tree management of a TODO-list. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-23Merge branch 'jk/mediawiki-credential'Libravatar Junio C Hamano1-28/+4
* jk/mediawiki-credential: mw-to-git: use git-credential's URL parser credential: convert "url" attribute into its parsed subparts mw-to-git: check blank credential attributes via length docs/credential: minor clarity fixups
2012-07-18mw-to-git: use git-credential's URL parserLibravatar Jeff King1-28/+4
We can just feed our URL straight to git-credential and it will parse it for us, saving us some code. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-18mw-to-git: check blank credential attributes via lengthLibravatar Jeff King1-1/+1
When writing a credential to git-credential, we omit fields that do not have a true value. This will skip empty or undefined fields (which we want), but will also accidentally skip usernames or passwords which happen to have a non-true value (e.g., "0"). Be more careful by checking for non-zero length. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-17git-remote-mediawiki: allow page names with a ':'Libravatar Matthieu Moy1-16/+33
Traditionnally, pages named Foo:Bar are page 'Bar' in namespace 'Foo'. However, it is also possible to call a page Foo:Bar if 'Foo' is not a namespace. In this case, the actual name of the page is 'Foo:Bar', in the main namespace. Since we can't tell with only the filename, query the wiki for a namespace 'Foo' in these cases, but deal with the case where no such namespace is found. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: properly deal with invalid remote revisionsLibravatar Matthieu Moy1-4/+9
Some wiki, including https://git.wiki.kernel.org/ have invalid revision numbers (i.e. the actual revision numbers are non-contiguous). Don't die when encountering one. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: show progress information when getting last remote ↵Libravatar Matthieu Moy1-0/+2
revision Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: show progress information when listing pagesLibravatar Matthieu Moy1-0/+3
Initial phases of push and pull with git-remote-mediawiki can be long on a large wiki. Let the user know what's going on. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: use --force when adding notesLibravatar Matthieu Moy1-1/+1
When notes are created to record a push, it normally doesn't exist yet. However, when a push is interrupted and then restarted, it may happen that a commit already has notes attached, and we want to reflect the newly created remote revision, hence use 'git notes add -f' to override the existing one Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: get rid of O(N^2) loopLibravatar Matthieu Moy1-7/+17
The algorithm to find a path from the local revision to the remote one was calling "git rev-list" and parsing its output N times. Run rev-list only once, and fill a hashtable with the result to optimize the body of the loop. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: make mediafiles export optionalLibravatar Matthieu Moy1-2/+22
It is possible to use git-remote-mediawiki on a tree with both .mw files and other files. Before git-remote-mediawiki learnt how to export mediafiles, such mixed trees allowed the user to maintain both the wiki and other files for the same project in the same repository. With the newly added support for exporting mediafiles, pushing such mixed trees would upload unrelated files as mediafiles, which may not be desired. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: actually send empty comment when they're emptyLibravatar Matthieu Moy1-1/+8
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-16git-remote-mediawiki: don't split namespaces with spacesLibravatar Matthieu Moy1-1/+1
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-13Merge branch 'mm/mediawiki-tests'Libravatar Junio C Hamano1-26/+123
* mm/mediawiki-tests: git-remote-mediawiki: be more defensive when requests fail git-remote-mediawiki: more efficient 'pull' in the best case git-remote-mediawiki: extract revision-importing loop to a function git-remote-mediawiki: refactor loop over revision ids git-remote-mediawiki: change return type of get_mw_pages git-remote-mediawiki (t9363): test 'File:' import and export git-remote-mediawiki: support for uploading file in test environment git-remote-mediawiki (t9362): test git-remote-mediawiki with UTF8 characters git-remote-mediawiki (t9361): test git-remote-mediawiki pull and push git-remote-mediawiki (t9360): test git-remote-mediawiki clone git-remote-mediawiki: test environment of git-remote-mediawiki git-remote-mediawiki: scripts to install, delete and clear a MediaWiki
2012-07-13Merge branch 'mm/mediawiki-file-attachments'Libravatar Junio C Hamano1-64/+386
"mediawiki" remote helper (in contrib/) learned to handle file attachments. * mm/mediawiki-file-attachments: git-remote-mediawiki: improve support for non-English Wikis git-remote-mediawiki: import "File:" attachments git-remote-mediawiki: split get_mw_pages into smaller functions git-remote-mediawiki: send "File:" attachments to a remote wiki git-remote-mediawiki: don't "use encoding 'utf8';" git-remote-mediawiki: don't compute the diff when getting commit message
2012-07-08git-remote-mediawiki: update comments to reflect credential supportLibravatar Matthieu Moy1-6/+1
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-06git-remote-mediawiki: be more defensive when requests failLibravatar Matthieu Moy1-0/+8
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-06git-remote-mediawiki: more efficient 'pull' in the best caseLibravatar Matthieu Moy1-21/+93
The only way to fetch new revisions from a wiki before this patch was to query each page for new revisions. This is good when tracking a small set of pages on a large wiki, but very inefficient when tracking many pages on a wiki with little activity. Implement a new strategy that queries the wiki for its last global revision, queries each new revision, and filter out pages that are not tracked. Signed-off-by: Simon Perrat <simon.perrat@ensimag.imag.fr> Signed-off-by: Simon CATHEBRAS <Simon.Cathebras@ensimag.imag.fr> Signed-off-by: Julien KHAYAT <Julien.Khayat@ensimag.imag.fr> Signed-off-by: Charles ROUSSEL <Charles.Roussel@ensimag.imag.fr> Signed-off-by: Guillaume SASDY <Guillaume.Sasdy@ensimag.imag.fr> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-06git-remote-mediawiki: extract revision-importing loop to a functionLibravatar Matthieu Moy1-3/+13
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-06git-remote-mediawiki: refactor loop over revision idsLibravatar Matthieu Moy1-6/+11
Without changing the behavior, we turn the foreach loop on an array of revisions into a loop on an array of integer. It will be easier to implement other strategies as they will only need to produce an array of integer instead of a more complex data-structure. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-06git-remote-mediawiki: change return type of get_mw_pagesLibravatar Matthieu Moy1-3/+5
The previous version was returning the list of pages to be fetched, but we are going to need an efficient membership test (i.e. is the page $title tracked), hence exposing a hash will be more convenient. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-04git-remote-mediawiki: improve support for non-English WikisLibravatar Matthieu Moy1-79/+61
Mediafiles can live in namespaces with names different from Image and File. While at it, rework the code to make it simpler and easier to read. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-27git-remote-mediawiki: import "File:" attachmentsLibravatar Pavel Volek1-8/+249
Add the symmetrical feature to the "File:" export support in the previous patch. Download files from the wiki as needed, and feed them into the fast-import stream. Import both the file itself, and the corresponding description page. Signed-off-by: Pavel Volek <Pavel.Volek@ensimag.imag.fr> Signed-off-by: NGUYEN Kim Thuat <Kim-Thuat.Nguyen@ensimag.imag.fr> Signed-off-by: ROUCHER IGLESIAS Javier <roucherj@ensimag.imag.fr> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-27git-remote-mediawiki: split get_mw_pages into smaller functionsLibravatar Matthieu Moy1-44/+62
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-27git-remote-mediawiki: send "File:" attachments to a remote wikiLibravatar NGUYEN Kim Thuat1-8/+90
The current version of the git-remote-mediawiki supports only import and export of plain wiki pages. This patch adds the functionality to export file attachments (i.e. the content of the File: MediaWiki namespace), which are also exposed by MediaWiki API. This requires a recent version of MediaWiki::API (Version 0.37 works. Version 0.34 doesn't). Signed-off-by: Pavel Volek <Pavel.Volek@ensimag.imag.fr> Signed-off-by: NGUYEN Kim Thuat <Kim-Thuat.Nguyen@ensimag.imag.fr> Signed-off-by: ROUCHER IGLESIAS Javier <roucherj@ensimag.imag.fr> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-27git-remote-mediawiki: don't "use encoding 'utf8';"Libravatar Matthieu Moy1-3/+2
The use of this statement is generally discouraged, and is too intrusive for us: it forces the HTTP requests made by the API to contain only valid UTF-8 characters. This would break the upload of binary files. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-27git-remote-mediawiki: don't compute the diff when getting commit messageLibravatar Matthieu Moy1-2/+2
While we're there, simplify the code a bit: since log --format=%s anyway shows the subject line as a single line, no need to split to take the first line. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-06-25git-remote-mediawiki: add credential supportLibravatar Matthieu Moy1-12/+94
The previous version implemented the possibility to log in a wiki, but the username and password had to be provided as configuration variables. We add the possibility to use the Git credential system to prompt the password. The support if implemented with generic functions that mimic the C API, designed to be usable from other contexts in the future (i.e. they may migrate to Git.pm if someone is interested). While we're there, do a bit of refactoring in mw_connect_maybe. Based on patch by: Javier Roucher Iglesias <Javier.Roucher-Iglesias@ensimag.imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-20git-remote-mediawiki: don't include HTTP login/password in authorLibravatar Matthieu Moy1-0/+4
On the MediaWiki side, the author information is just the MediaWiki login of the contributor. The import turns it into login@$wiki_name to create the author's email address on the wiki side. But we don't want this to include the HTTP password if it's present in the URL ... Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-28git-remote-mediawiki: allow a domain to be set for authenticationLibravatar Matthieu Moy1-0/+3
When the wiki uses e.g. LDAP for authentication, the web interface shows a popup to allow the user to chose an authentication domain, and we need to use lgdomain in the API at login time. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-27git-remote-mediawiki: obey advice.pushNonFastForwardLibravatar Matthieu Moy1-7/+10
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-27git-remote-mediawiki: set 'basetimestamp' to let the wiki handle conflictsLibravatar Matthieu Moy1-6/+37
We already have a check that no new revisions are on the wiki at the beginning of the push, but this didn't handle concurrent accesses to the wiki. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-27git-remote-mediawiki: trivial fixesLibravatar Matthieu Moy1-2/+1
Fix a whitespace issue (no space before :) and remove unused %status in mw_push. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-01git-remote-mediawiki: allow push to set MediaWiki metadataLibravatar Matthieu Moy1-2/+33
Push can not set the commit note "mediawiki_revision:" and update the remote reference. This avoids having to "git pull --rebase" after each push, and is probably more natural. Make it the default, but let it be configurable with mediawiki.dumbPush or remote.<remotename>.dumbPush. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-01Add a remote helper to interact with mediawiki (fetch & push)Libravatar Jeremie Nikaes1-0/+756
Implement a gate between git and mediawiki, allowing git users to push and pull objects from mediawiki just as one would do with a classic git repository thanks to remote-helpers. The following packages need to be installed (available on common repositories): libmediawiki-api-perl libdatetime-format-iso8601-perl Use remote helpers in order to be as transparent as possible to the git user. Download Mediawiki revisions through the Mediawiki API and then fast-import into git. Mediawiki revision number and git commits are linked thanks to notes bound to commits. The import part is done on a refs/mediawiki/<remote> branch before coming to refs/remote/origin/master (Huge thanks to Jonathan Nieder for his help) We use UTF-8 everywhere: use encoding 'utf8'; does most of the job, but we also read the output of Git commands in UTF-8 with the small helper run_git, and write to the console (STDERR) in UTF-8. This allows a seamless use of non-ascii characters in page titles, but hasn't been tested on non-UTF-8 systems. In particular, UTF-8 encoding for filenames could raise problems if different file systems handle UTF-8 filenames differently. A uri_escape of mediawiki filenames could be imaginable, and is still to be discussed further. Partial cloning is supported using one of: git clone -c remote.origin.pages='A_Page Another_Page' mediawiki::http://wikiurl git clone -c remote.origin.categories='Some_Category' mediawiki::http://wikiurl git clone -c remote.origin.shallow='True' mediawiki::http://wikiurl Thanks to notes metadata, it is possible to compare remote and local last mediawiki revision to warn non-fast forward pushes and "everything up-to-date" case. When allowed, push looks for each commit between remotes/origin/master and HEAD, catches every blob related to these commit and push them in chronological order. To do so, it uses git rev-list --children HEAD and travels the tree from remotes/origin/master to HEAD through children. In other words: * Shortest path from remotes/origin/master to HEAD * For each commit encountered, push blobs related to this commit Signed-off-by: Jérémie Nikaes <jeremie.nikaes@ensimag.imag.fr> Signed-off-by: Arnaud Lacurie <arnaud.lacurie@ensimag.imag.fr> Signed-off-by: Claire Fousse <claire.fousse@ensimag.imag.fr> Signed-off-by: David Amouyal <david.amouyal@ensimag.imag.fr> Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr> Signed-off-by: Sylvain Boulmé <sylvain.boulme@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-03 17:54:58 +0000