Fremantle2026 April 21 (Tuesday), 11:19AM· Internet Archive · backups · MediaWiki · I now have a fresh 10 TB on my desk to fill up with various things I've been uploading to the Internet Archive. I'm not really sure about keeping all the annual dumps of wikis, but then I might as well. I do want to think about a better way to get MediaWiki images/ directories onto IA, because at the moment each dump contains a lot of stuff that's in previous ones. I'm pondering some clever system of splitting images/ into 500-file chunks (as the limit for individual items). The problem is that it'd be nicest if new files could be appended to the most recent item (rather than having to shuffle files between items). Or maybe the duplication just doesn't matter, and a new one- or two-hundred gig dump every year is acceptable. Just feels inefficient to me. This post is also on Mastodon. ← Previous My main RSS news feed: https://samwilson.id.au/news.rss(or Wikimedia.rss, Fremantle.rss, OpenStreetMap.rss,…
No comments yet. Log in to reply on the Fediverse. Comments will appear here.