What is Reddit's opinion of
Kiwix?
From 3.5 billion Reddit comments
56 reviews of this app found across Reddit:
- Download Kiwix
- Load it up with the English Wikipedia and Project Gutenberg
- Done
Btw, I own a Casio Denshi Jisho and most of the reference works are highly specialized, such as one just for medicines, or one just for agriculture. The novels feature is decent though. It's also loaded with some classical music. And even famous speeches; mine has one from Obama.
You guys know it, Kiwix is an offline reader that allows users to browse entire copies of Wikipedia (ca. 83Gb for the whole thing, incl. images), StackExchange (new release coming up soon), the Gutenberg project, etc. stored locally. The project is fully FOSS, lives from user donations, and the subreddit is r/Kiwix.
The PR has been made last night so consider it fresh off the press. Feel free to test and report issues here. Thanks to the folks who made it possible.
You can download the HTML files but it's a pretty difficult way to navigate through it. You can download a file and open it through Kiwix which provides a pretty good way to navigate through it.
Awesome job Kiwix team! You are peak datahoarder material!
Your work has helped so many people access information, and allowed us datahoarders to archive these excellent resources in an accessible way.
For those of you that don't know what this project is: it is a way to archive and access offline archives of Wikipedia, Wiktionary, TED conferences, the Gutenberg library, Stack Exchange, and some others as well. Here's their full list
This is an awesome project, and if you have the ability, I highly recommend you use the stuff this project has worked on, tell your friends, and drop them a donation to support their hard work!
I'm a data hoarder, primarily focusing on software, books, comics, and pen-and-paper tabletop RPGs. I've got north of 16TB of stuff so far, covering every range of subjects but especially focusing on skills that would be useful if a cyber attack or EMP were to destroy our grid. Such an eventuality would kill tens of thousands if not resolved in a few days, millions in a few months, and ~85% of the population in a year. My setup is portable, kept on a few small SSDs in an EMP and water-proof bag, along with a radio, laptop, solar cells, batteries, and an inverter.
You don't have to be as extreme as I am. You can start small with a flash drive and offline copy of Wikipedia. If you want to go beyond this, check out r/DataHoarder.
Btw, kamu bisa ngedownload wikipedia dalam bentuk ter-compress sbg file zim dan bisa dibuka secara offline dengan program kiwix. Ini bagus untuk daerah dengan koneksi minim, layaknya proyek lain seperti "internet in a box"
>not only could I not download the entirety of wikipedia without paying but the site also said I couldnt browse wikipedia locally on the raspberry pi.
I don't know which site they sent you to, but Kiwix is free, always has been and always will be (that's what free software is about): it only relies on donations. The installer's user manual is here.
The one configuration where you could be asked to pay is if you want the Raspbian image to be built on Kiwix servers rather than your own computer (saves a lot of time).
Kiwix is already available on macOS but it's managed by a lone (yet awesomely dedicated) volunteer so while it's great and does most of the job it is clearly not on par with other versions (I think he's more focused on the iOS port).
As for Synology, this came straight from them so I'm not sure what their plans are.
Don't waste your time trying to pick through Wikipedia articles. Just download all of Wikipedia.
You can use Kiwix or XOWA to download all of wikipedia, I have heard Kiwix is easier but I used XOWA.
When compressed, it's small enough to fit on a flash drive. And then you will never have a moment where you say "Oh shit, I should have saved that article about how to make prosciutto because the world has collapsed and the knowledge is lost!" because you will have a backup of everything.
FYI if you want something that's ready to use, Kiwix has prebuilt, compressed, searchable copies of Wikipedia and a bunch of other sites (like StackOverflow) and viewer apps for Linux, Android, and all the peasant platforms too.
I'm surprised I haven't seen Kiwix recommended yet. It's got StackOverflow and a buuuunch of other stuff available for download and I've used it myself - works quite well, even searching! (And you can host it as a webapp if you want.)
That actually is the most recent English maxi release, as they are having some issues with it.
https://www.reddit.com/r/DataHoarder/comments/fguv9d/where_is_wikipedias_latest_kiwix_zim_files/
https://www.kiwix.org/en/whats-wrong-with-the-english-wikipedia/
You can also check the status of the scraper here - https://farm.openzim.org/recipes/wikipedia_en_all_maxi
Please don't download that, its not readable like you think it might be. The data dumps are like massive databases and not like click to open pdf's.
​
You will need a program like kiwik to read Wikipedia pages offline. https://www.<strong>kiwix</strong>.org
Except for the weather part it looks very much like what the Kiwix hotspot has to offer. There are QR codes generators available out there and the default address is http://goto.kiwix.hotspot (or name.hotspot).
The nimble could also be it, albeit a lot more sophisticated (the whole tech is open-source so you should be able to find blueprints if you don't want to purchase one).
In any case the Pi would be your way forward, it only needs 5V/2A so a $30 solar panel would be sufficient to power it.
So probably PDF isn't the most efficient way to store wikipedia. I think you might be shocked at how large that would be. A PDF of the dolphin wikipedia page weighs in at about 10 MB, and you can extrapolate from there.
What you really want is an offline wikipedia database, most likely, and these already exist. You might want to check out Kiwix: https://www.kiwix.org/
SEO is good but we need to clarifiy the copy. We're a non-profit with two main, unfortunately non-overlapping objectives: increase donors on the one hand, attract more paid customers for the (paid) Raspberry hotspot offer on the other (which channels for the latter would be useful; we get 10k of free Google ads each month that are largely underused or do not convert to actual sales).
Thanks!
Do you have any sort of actual response? My entire point is that its useful when you dont have internet and cant check the actual sites because of that. I'm not trying to argue that everyone should go back to downloaded content and minimize use of the internet, thats something you came up yourself, and I don't know where the hell you've come up with the whole control thing. Again if you have no use for it, thats fine, but you are sheltered if you think that nobody has a usecase for it, because you are unable to understand the conditions that would make it useful. The entire reason kiwix is making these dumps in the first place is for the 40% of the world that has no internet access, period. (And yes, thats not the exact same thing as what I was talking about inb4 you try to jump on that)
Ill say it one more time: Nobody is trying to make you do anything, I am simply making the case for why it is sometimes useful to some people.
Sorry for what you are going to endure.
The value of knowledge and information is not negligible in a country where censorship is state-led. You can download the Wikipedia Edition you want. Maybe download the English one to improve your english too! :-)
Try to download Kiwix, an offline reader. https://www.kiwix.org/en/
Every steps are explained on their wiki. https://wiki.kiwix.org/wiki/Main_Page
You can chose the projet and the edition (language). When in doubt, I would download everything but Wikinews and Wikivoyage...
The English edition of Wikipedia is about 8GB without any media for more than 6 million articles.
Kiwix is a raspberry Pi based device that can host offline versions of Wikipedia, Project Gutenberg … etc and create a wifi hotspot to broadcast to mobile devices. It’s used for remote, rural classrooms. Bring a dozen iPads with you in a charging dock case, and a deep cycle solar charging inverter station Make sure the iPads also have the Zygote Body Human Anatomy Atlas loaded on it, a copy of “Where there is No Doctor”. An offline cache download of Google Earth, Machinery’s Handbook, Pocket Ref… and the complete works of Hanna Barbera.
I should say I have the my local Danish Wikipedia on my phone since it only takes 2 GB where the English is 87GB full, but they come in many versions
- Kiwix is the app to read the database files like a browser
- Database files in all languages and types of specialisation.
- When you open the app you point to the offline database file and add it.
You should check out Kiwix (specifically Hotspot https://www.kiwix.org/en/documentation/how-to-set-up-kiwix-hotspot/ ). You select all the stuff you want to add to a microsd (wikipedia, ted talks, stack exchanges, tons of options), and it builds a Pi image that creates a hotspot for you to connect to and get into it all! And you can run it off of just a Pi Zero W. All English Wikipedia (including photos) is ~80gb - so as you can imagine you could fit a ton on larger SD cards!
Plus they do good work - they provide these to areas with little or no internet access!
Full link is here: https://www.kiwix.org/en/what-to-expect-from-this-years-google-summer-of-code-kiwix/
And why TF would Reddit put the donate picture as default thumbnail?
I think you can use kiwix (https://www.kiwix.org/en/) to basically download a bunch of things offline (Wikipedia, stackexchange). The download can be selective, e.g. you can download the chemistry stuff of Wikipedia only and the latex in stackexchange. Maybe downloading it will also serves you well. I never used it, but maybe you can give it a try.
Not true. The 80 GB Wikipedia copy on my phone has images. They're not original resolution, but they're still good enough for most things.
> Seems like it would be awesome to setup a little RPi hotspot for like van lifers and the like.
Glad you're asking! Here is our RPi image creator (looking for a cooler name than "Hotspot installer", feel free to pitch in)
> I deleted those files (for no particular reason)
I think we might be onto something already. If you haven't run any update on Kiwix, then maybe another install you did during that time might have caused conflicts. Have you tried to reinstall Kiwix?
What device are you on (android? Windows? Linux? iOS or macOS)? What were the files that you deleted? What exact version of Wikipedia were you looking at (language, variant (maxi/nopic), and date)?
There's a blog post on how to open a good issue: https://www.kiwix.org/en/how-to-report-a-bug/
Kiwix is mostly run by volunteers, so it really helps to help them with as much info as possible from the outset.
The numbers are huge if you want all the edit history, but it you just want the current text of all the articles it's just a few GB- easily fitting on a USB drive. Look into the Kiwix project which distributes language versions of Wikipedia and other free sites on devices: https://www.kiwix.org/en/
That's kind of what this is to me. I'm building a bigger one for daily use soon and this one will sit in a faraday cage in the garage. You can get wiki down to 39gb as well if you don't include images or download simple english. Theres actually an insane amount of content you can serve offline, check it out here
I believe you can set it to that but it would require copying a new daily to your server, when you're dealing with a file of this size that's a big ask. Speaking of file size though, its only 99gb for all of wikipedia on 10/28/2020 including images. This open source software project called Kiwix has developed this pretty insane compression algorithm called .Zim specifically for storing and serving large websites offline. Pretty dope. https://www.kiwix.org/en/
You can use https://www.kiwix.org, outside of the full wikipedia dump it has several collections so you can get the top 5000 articles, or only subjects you care about. It also support other sources like StackOverflow.
I get the ' wikipedia_en_all_maxi_XXX' from here:
https://download.kiwix.org/zim/wikipedia/
Use the Kiwix tools to serve it Here:
https://www.kiwix.org/en/downloads/kiwix-serve/
So no need to uncompressed.
If you are serving with windows just make sure you use ver i686-3.1.2-3 or greater else it will report the ' wikipedia_en_all_maxi_2020-09.zim ' 93gb file as corrupt.
I had an older version serving a 2018 wikipedia_en_all_maxi. Wasted much time with the new .zim file until I found the newer 3.1.x version.
To serve a single file:
kiwix-serve --port=8087 D:\Kiwix\Zim_Files\wikipedia_en_all_maxi_2020-06.zim
For Example .. using the tools you can build a Lib:
Add to Lib - If nothing then will create
kiwix-manage library_zim.xml add D:\Kiwix\Zim_Files\wikipedia_en_all_maxi_2020-07.zim
kiwix-manage library_zim.xml add D:\Kiwix\Zim_Files\askubuntu.com_en_all_2020-01.zim
kiwix-manage library_zim.xml add D:\Kiwix\Zim_Files\electronics.stackexchange.com_en_all_2020-01.zim
Show Lib Items
kiwix-manage library_zim.xml show
Serve up Lib files:
kiwix-serve --port=8087 --library D:\Kiwix\Zim_Files\library_zim.xml
Have fun...
>A PDF format seems preferable for me.
Are you sure about this? If you're set on offline use wouldn't you still prefer to have access to Wikipadia's interlinked structure? With 1000 separate PDFs you'd have to manually construct your own reference points and indexes to get from one article to the next.
Using Kiwix instead will allow you to access an offline version of Wikipadia. What content you can access depends on the ZIM-file you use. Creating custom ZIMs about only Electricity-related articles might be a bit tricky but since the whole English Wikipedia amounts to "only" about 92GB that might just be the simpler solution if you have the space to spare.
I downloaded Kiwix and then through Kiwix the whole of Wikipedia. It's 80 GB, but it's well worth it to have a local copy. Also through Kiwix I downloaded all the English books on Project Gutenberg, at 50 GB. There's much more as well.
There's a RPi installer if that's what you are looking for and it works on all of them (W -> 4)
https://www.kiwix.org/en/downloads/kiwix-hotspot/
There are plans so make premade images available for download - ideally before the end of this month. Ping me if you want me to ping you when it's out (or simply subscribe to r/Kiwix as the announcement will first go there)
Offline Wikipedia: <strong>Kiwix</strong>
Available for Android, iPhone, Mac, Windows and Linux
Allows for download of Wikipedia offline in various sections or altogether, as well as many different StackExchanges.
Can also host on a network and allow other on the network to access the knowledge.
Essential for rebuilding
Download wikipedia for offline use: https://www.kiwix.org/en/
Put it on a spare backup phone or tablet, along with photos of family and friends, and as many useful books you can find. Keep the battery 50% charged for storage. Check on it like you would any other perishable item and refresh with new items.
The Quad-core 1.9 GHz processor is still plenty fast. There's a few emulators that would work much better if it had 64-bit support, but other than that I'd rather have similar-speed and longer-battery-life processor than a similar-battery-life and faster processor. My battery is about a year and a half old, and starting to degrade, but still usually lasts a whole day of moderate use. I'll need to replace it soon, but that only costs $20, and takes a few seconds.
I have a 256 GB SD card in it, because I keep a copy of Wikipedia on it, which gives me most of the information I would ever need in an emergency, even when I don't have a cellular signal. There's plenty of room leftover, so storage space isn't an issue.
I'm looking into building my own kernel, so I can always have the latest patches, which would make my Note 4 more secure than the latest Note 10.
Thanks for all the information. Bookmarked for later. :)
You seem pretty plugged-in to the community… Do you know if anyone’s managed to get Kiwix to work on the Kobo yet?
When I first got my Kobo I looked into that—offline Wiki would be very useful!—but the sites I was finding seemed to be perpetually in development.
You can get a cheap 100gb flash drive off Amazon and put the full dump of Wikipedia and images on it, it'd come in handy if you're trying to protect a USB drive in the first place.
Here are some instructions I gave to someone else a while ago
INSTRUCTIONS
https://www.kiwix.org/en/downloads/
You'll want to download a reader like kiwix (the link) and when you open up the application, there should be 2 options somewhere in the upper left, pick the one that isn't looking through "Local"
Then you can choose what information you want offline (for example there's a StackExchange dump) just sort by size and pick the second one which should be a 78GB Wikipedia dump, if the download within the app doesn't work you can go back to the kiwix site and find the torrent for the Wikipedia dump under "content packages" section where there's a link to the wiki, pay attention to the size and language when looking through them
Kiwix has been available to do just that for a while, and not only Android/iOS but also Desktop and Raspberry Pi.
It's also fully open-source (feel free to donate your $9 though).
You should look at getting a cheap android device, as there are quite a few full sized wikipedia conversions including graphics available for that platform. You would need a very large microsd card too.
Yes, kiwix is very good.
https://www.kiwix.org/en/downloads/kiwix-reader/
It uses the ZIM file format.
Wikipedia in English is still failing to update. We are, indeed, on it.
There's a bit more info here: https://www.kiwix.org/en/whats-wrong-with-the-english-wikipedia/
Download a Kiwix set with images onto your NAS. Setup a Kiwix browser on the devices you want to use. Point it to the share address.
Your question sounds like you want to run a local offline copy of Wikipedia that you can login to from a web browser and just browse as normal. There are web apps that let you browse Wikipedia database dumps I assume, but it's not going to be the full 100% Wikipedia experience. The database dumps with images just include the thumbnails.
https://www.kiwix.org/en/downloads/
You'll want to download a reader like kiwix (the link) and when you open up the application, there should be 2 options somewhere I. The upper left, pick the one that isn't looking through "Local"
Then you can choose what information you want offline (for example there's a StackExchange dump) just sort by size and pick the second one which should be a 78GB Wikipedia dump, if the download within the app doesn't work you can go back to the kiwix site and find the torrent for the Wikipedia dump under "content packages" section
Hey thanks but that's a pretty old article (2013) listing pretty old versions of the software. Might as well point people directly to the current ones: https://www.kiwix.org/en/downloads/kiwix-reader/
Kiwix is a good offline wiki reader. The whole of english wikipedia (with images, no video) is about 78GB. I have it on my tablet. They have heaps of other stuff linked too which is available.
Kiwix is an offline wikipedia application that works on phones (Android & iOS) and PCs https://www.kiwix.org/en/
The whole English Wikipedia with images is about 79 GB right now according to the zim file list here https://wiki.kiwix.org/wiki/Content_in_all_languages, hope you have a big microSD card.
Please don't crawl Wikipedia with HTTrack. It's a huge waste of bandwidth when there are numerous backups you can just download directly. Wikimedia even releases snapshots themselves.
Check out Kiwix. That's the easiest way to do it and won't use up dozens of gigabytes of Wikimedia's bandwidth to reinvent the wheel. https://www.kiwix.org/en/home/
OK, sorry for not explaining properly, I don't have a technical background, what I mean is the same functionality as this Raspberry Pi box but on a phone
https://www.kiwix.org/en/downloads/kiwix-hotspot/
The task I linked to describes this as a Kiwix HTTP server, hope this makes sense
Check out this post on the Wikimedia blog where a group did that exactly: https://wikimediafoundation.org/2019/02/18/to-bridge-perus-digital-divide-these-researchers-are-taking-wikipedia-offline/. They're using what they call an internet-in-a-box, and the site includes build instructions (it runs on a Raspberry Pi). I think the underlying software is Kiwix Server.
These .Zim files for stack exchange and Wikipedia should be downloaded and seeded via torrent on all the big mesh systems like Yggdrasil. I'd love to see those services have enough torrent DHT nodes to be sustainable without the internet.
Also, the Ubuntu/Kubuntu and Raspbian repositories, and the popular Arduino tools, and Python stuff.
Kubuntu because it's a distro you can hand a Windows user who doesn't really like computers, and they'll be able to pick it up right away.
Raspbian and Arduino because it's almost the global standard for really quickly building embedded systems for anyone from hobby to pro, unless they have enough time to do more custom stuff.
Also, Zeal docsets, because programming languages are useless without documentation and everyone relies way too much on online docs that can be downloaded instead.
HiJacking the popular thread since literally nobody bothered show how to use Wikipedia offline.
Grab Kiwix for your OS and then select which database you'd like: https://www.kiwix.org/downloads/
Wikipedia is 35 gigs with no pictures and 79 gigs with no videos and compressed images.
If you know what you're doing you can also download most Wikimedia content from here: https://dumps.wikimedia.org/
Wikipedia when highly compressed can fit in as little as 15.3 gigs.