The Longest Now

The last available dump of the entire English Wikipedia is 10 weeks old.  Many dumps have been attempted since then, all have failed.  There is currently no way for an individual to sign up for incremental updates from the site, so the only option for updating one’s local dump/mirror/archive/testing-ground is to retrieve the entire thing from download.wikimedia.org.

XML dumps are not available from Wikimedia servers; you have to get a massive tarball — even though many people and most researchers will convert this to xml before doing anything further with it.  Other useful but unavailable dumps include: randomized subsets of en:wp and other languages, a dump of all commons images, a dump of all images used on en:wp or other language wp’s, whether on commons or not, a dump of any media uploaded in the past 11 months (latest dump here).  Worth noting: the images/ subdirectory on download.wikimedia.org isn’t officially linked to from anywhere.

There are limits to use of information.  One is free copyright, one is free format, but before all of those is discovery and access to any version at all.  I hope these oversights are remedied soon.

Wikipedia content *sometimes* available …