Wikix - Revision history

wikademia>Wikademia at 21:16, 20 June 2009

2009-06-20T21:16:43Z

← Older revision		Revision as of 21:16, 20 June 2009
Line 1:		Line 1:
	'''Wikix''' is a 'C' based program written by Jeffrey Vernon Merkey that will read any XML dump provided by the foundation, extract all image names from the XML dump which it may reference, then generate a series of BASH or Bourne Unix style scripts which can be invoked to download all images from ''Wikimedia Commons'' and ''Wikipedia''.		'''Wikix''' is a 'C' based program written by Jeffrey Vernon Merkey that will read any XML dump provided by the foundation, extract all image names from the XML dump which it may reference, then generate a series of BASH or Bourne Unix style scripts which can be invoked to download all images from ''Wikimedia Commons'' and ''Wikipedia''.
			{{?}}
	The program relies on [[:en:cURL\|cURL]], an automated web spider, to download referenced images. The program will also convert text based utf8 characters into actual utf8 strings for those dumps which may contain improperly formatted names for specific images. The program can be configured to generate 16 parallel scripts which will download all images from Wikipedia. The program includes Jeff Bezanson's utf8 libraries.		The program relies on [[:en:cURL\|cURL]], an automated web spider, to download referenced images. The program will also convert text based utf8 characters into actual utf8 strings for those dumps which may contain improperly formatted names for specific images. The program can be configured to generate 16 parallel scripts which will download all images from Wikipedia. The program includes Jeff Bezanson's utf8 libraries.

wikademia>Wikademia: 31 revisions

2009-04-19T18:37:40Z

31 revisions

← Older revision	Revision as of 18:37, 19 April 2009
(No difference)

wikademia>American Eagle: Undo revision 1424688 by 68.87.42.115 (talk)

2009-03-13T23:09:29Z

Undo revision 1424688 by 68.87.42.115 (talk)

← Older revision		Revision as of 23:09, 13 March 2009
Line 15:		Line 15:
	#Now type in your terminal.<br><pre>make</pre>Now if the compilation and linking of Wikix completes without errors then you will have a brand new executable - ''wikix'', in your ''/home/you/wikix'' (in this example) directory.		#Now type in your terminal.<br><pre>make</pre>Now if the compilation and linking of Wikix completes without errors then you will have a brand new executable - ''wikix'', in your ''/home/you/wikix'' (in this example) directory.

	In case of any problems, please report it in the [[Talk:Wikix\|discussion]] page.~~test~~		In case of any problems, please report it in the [[Talk:Wikix\|discussion]] page.

	== Options ==		== Options ==

68.87.42.115: /* Compiling and Installation */

2009-03-13T19:54:36Z

Compiling and Installation

← Older revision		Revision as of 19:54, 13 March 2009
Line 15:		Line 15:
	#Now type in your terminal.<br><pre>make</pre>Now if the compilation and linking of Wikix completes without errors then you will have a brand new executable - ''wikix'', in your ''/home/you/wikix'' (in this example) directory.		#Now type in your terminal.<br><pre>make</pre>Now if the compilation and linking of Wikix completes without errors then you will have a brand new executable - ''wikix'', in your ''/home/you/wikix'' (in this example) directory.

	In case of any problems, please report it in the [[Talk:Wikix\|discussion]] page.		In case of any problems, please report it in the [[Talk:Wikix\|discussion]] page.test

	== Options ==		== Options ==

wikademia>Jlerner: /* Options */ new section

2009-01-09T20:03:30Z

Options: new section

← Older revision		Revision as of 20:03, 9 January 2009
Line 16:		Line 16:

	In case of any problems, please report it in the [[Talk:Wikix\|discussion]] page.		In case of any problems, please report it in the [[Talk:Wikix\|discussion]] page.

			== Options ==
			<pre>
			# ./wikix -h
			USAGE: wikix -htrciop < file.xml [ > script.out ]
			-h this help screen
			-t use xml dump to strip from tree
			-r wikipedia path
			-c commons path
			-i image path
			-o output path
			-p parallel (16 process) mode
			</pre>

	== Example ==		== Example ==

wikademia>Jlerner: /* Example */ how to import the images

2009-01-09T20:02:25Z

Example: how to import the images

← Older revision		Revision as of 20:02, 9 January 2009
Line 19:		Line 19:
	== Example ==		== Example ==

	The program would typically be invoked in a directory that you wish to use to host the images. Wikix will construct a MediaWiki style directory structure which can be quickly imported into a MediaWiki Wikipedia installation:		The program would typically be invoked in a directory that you wish to use to host the images. Wikix will construct a MediaWiki style directory structure which can be quickly imported into a MediaWiki Wikipedia installation (e.g., via <tt>php rebuildImages.php --missing</tt>):

	<pre>		<pre>

62.2.213.2: fixing capitalization, linking

2008-07-22T13:28:57Z

fixing capitalization, linking

← Older revision		Revision as of 13:28, 22 July 2008
Line 1:		Line 1:
	'''Wikix''' is a 'C' based program written by Jeffrey Vernon Merkey that will read any XML dump provided by the foundation, extract all image names from the XML dump which it may reference, then generate a series of BASH or Bourne Unix style scripts which can be invoked to download all images from ''Wikimedia Commons'' and ''Wikipedia''.		'''Wikix''' is a 'C' based program written by Jeffrey Vernon Merkey that will read any XML dump provided by the foundation, extract all image names from the XML dump which it may reference, then generate a series of BASH or Bourne Unix style scripts which can be invoked to download all images from ''Wikimedia Commons'' and ''Wikipedia''.

	The program relies on ~~Curl~~, an automated web spider, to download referenced images. The program will also convert text based utf8 characters into actual utf8 strings for those dumps which may contain improperly formatted names for specific images. The program can be configured to generate 16 parallel scripts which will download all images from Wikipedia. The program includes Jeff Bezanson's utf8 libraries.		The program relies on [[:en:cURL\|cURL]], an automated web spider, to download referenced images. The program will also convert text based utf8 characters into actual utf8 strings for those dumps which may contain improperly formatted names for specific images. The program can be configured to generate 16 parallel scripts which will download all images from Wikipedia. The program includes Jeff Bezanson's utf8 libraries.

	As of March 24, 2008, using a cable modem, the entire set of Wikipedia images can be downloaded in about 96 hours using this program (420 GB as of 3/24/08).		As of March 24, 2008, using a cable modem, the entire set of Wikipedia images can be downloaded in about 96 hours using this program (420 GB as of 3/24/08).

149.99.42.22: /* Example */ small usage fix.

2008-06-16T15:55:13Z

Example: small usage fix.

← Older revision		Revision as of 15:55, 16 June 2008
Line 25:		Line 25:
	</pre>		</pre>

	The -p option tells wikix to create parallel scripts. If you omit the -p option, it will create one very large file. By default, the program is ~~setup~~ to mirror the English Wikipedia. You can override the default settings by substituting path information for commons and the target Wikipedia site through the command line options.		The -p option tells wikix to create parallel scripts. If you omit the -p option, it will create one very large file. By default, the program is set up to mirror the English Wikipedia. You can override the default settings by substituting path information for commons and the target Wikipedia site through the command line options.

	The program will create a series of scripts as:		The program will create a series of scripts as:

166.70.238.44: /* Source Code */

2008-05-01T19:15:59Z

Source Code

← Older revision		Revision as of 19:15, 1 May 2008
Line 83:		Line 83:
	The full source code with build scripts can be download from this ftp server:		The full source code with build scripts can be download from this ftp server:

	*ftp://www.~~wikigadugi~~.org/~~wiki~~/~~MediaWiki~~/wikix.tar.gz~~.bz2~~		*ftp://www.wolfmountaingroup.org/pub/wikix/wikix.tar.gz

	=== Source Code Files ===		=== Source Code Files ===

129.138.6.212: /* Example */

2008-05-01T05:15:46Z

Example

← Older revision		Revision as of 05:15, 1 May 2008
Line 22:		Line 22:

	<pre>		<pre>
	wikix -p < name_of_xml_file.xml &		wikix -p < name_of_xml_file.xml > script.out &
	</pre>		</pre>