Considering the tendency even now of LLMs to hallucinate, and the hardware requirements for any kind of LLM -why not just gather those resources into some sort of local archive that can be searched without an LLM?
This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
Not sure about how the code in this project works. But the way I see it, an LLM+chat is a great interface for information. If you then connect it to a db with the knowledge/articles you need, then you get a great way to navigate it, without needing the model to memorize all the info
I think that if I were to build this I might skip the LLM and just setup a nice interface to the source files using qmd style search. [0]
That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.
I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.
Awesome, I didn't know qmd style search, thank you!
I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)
I agree with the OP; this technology does not seem an appropriate fit for survival equipment. By definition survival scenarios can be life and death. Even one "You're absolutely right!" level mistake from an LLM could give a user bad information, leading to John Doe eating a mushroom he really should not have eaten.
I agree with your point, but if John Doe is foraging mushrooms without experience of the local flora (which I doubt this archive contains), then John Doe is kinda asking for it.
The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.
It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
It already is a local archive that can be searched independently, but the LLM does RAG and is, according to them, 95% accurate. It can be useful if you don’t remember where to find a source/what words to search for, but even a durable device with Kiwix would be useless in a a lot of survival situations people imagine.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.
> Mac or PC with 16–31 GB | IBM Granite 4.2 8B | 14 of 22 right | 6 GB
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Or just.. download the books, and slap a search engine on top of them. It'll be like 10x smaller than the model. "Survival LLM" is absolutely a non-relevant use case.
We had a set called something like “How to do anything in an emergency” which had instructions for this and more. Alongside lots of creative uses of potassium permanganate.
I’m pretty sure it also told you how to butcher a sheep.
I used to be obsessed with it as a kid. The spectre of nuclear war and a three volume handbook on how to survive the impending apocalypse with nothing more than a plastic bucket, a packet of potassium permanganate, some charcoal and string. The late 80s and early 90s were a wild time to be a kid.
No thanks, I'll use a generic model along with trusted sources.
Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.
Back in 2018 I was working on a project for educating schools in certain parts of Africa and Middle East, on AI. The issue was it had to be air-gapped (no internet guarantees) and small/portable. I ended up downloading Wikipedia (not all of it), and then created a lightweight inverted index (for search) and TextRank (for summaries). The whole thing was on a Raspberry Pi, and installed in a small plastic enclosure. We had many of these and could easily ship them. There was an "AI Learning Module" as well we built, that started with Scratch, and then moved into Python. But if I was to do it all over today, I would probably add a tiny LLM and maybe a small ASR and a TTS model too, and you basically have a completely self-contained AI knowledge assistant in a cigarette box. The point being, you basically also have a "survival AI" you can take with you. Of course you still need electricity...
Love this. And as a trend, I'd love to see more local/small task-specific models/agents. Experts at helping you with that one thing, leaving the everything-and-the-kitchen-sink to the frontier models/agents
On the setup process: "You only click for about 5 minutes; the rest is downloading." On the energy consumption: "A small generator or power station runs it easily"
Five minutes of clicking and a (small) power station is a small price to pay when survival could be at stake.
With all due respect: why does this need an LLM, and this much data? Surely a "classic" knowledge base with a more traditional search bar, running on top of the same data this LLM was trained on would be way smaller, and actually as trustworthy as the source material it surfaces for you?
There are a fair number of people who seem to believe that LLMs contain "world knowledge" and are blissfully unaware of how inaccurate an ungrounded chatbot can be. They ask it questions, it gives them confident and plausible (sometimes even correct!) answers, and they leap to the conclusion that these are useful tools for end-of-civilization knowledge bases. I fought this battle countless times on HN and Reddit, generally unsuccessfully.
This project at least attempts to ground its answers with (presumably) human-vetted sources. That will make it much, MUCH more accurate and truthful than a bare model. But there is still a non-zero chance for it to hallucinate answers, especially when posed a question that it doesn't have any source material in its database.
As for why bother hooking up an LLM instead of a simple search engine, search engines don't synthesize data from multiple sources into a coherent answer for you, but LLMs are (mostly) great at that.
That said, while I like this pattern (LLM + knowledge base) for lots of things, this isn't necessarily something I'd trust with my own (or my family's) survival.
It's disappointing that Gemma E4B seems to perform poorly on this (42% accuracy according to the GitHub). It would be nice if a $25 phone could be loaded up with this and stuffed in a EMP proof metal box with a small solar charger. Maybe a finetune or Gemma 5 will get us there.
If you're actually offline and need information in a 'survival' situation, I'm not so sure the hallucinatory nature of an LLM is the based way to distribute a repository of information. Additionally, there's no need to fit an LLM entirely in memory.
In a theoretical scenario where the internet is gone and isn't coming back: If you can deal with a very low token/second rate there's no reason not to store 'offline' a much larger model like qwen 3.8-flash-next or deepseek v4 flash 1017 and use an inference engine which can stream-read it off disk to extract the information you want, albeit slowly. Of course also download and use things that'll fit in RAM, sure, like the qwen 3.8 27B in the linked page. But no need to restrict yourself to only smaller models.
Considering the tendency even now of LLMs to hallucinate, and the hardware requirements for any kind of LLM -why not just gather those resources into some sort of local archive that can be searched without an LLM?
This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
Not sure about how the code in this project works. But the way I see it, an LLM+chat is a great interface for information. If you then connect it to a db with the knowledge/articles you need, then you get a great way to navigate it, without needing the model to memorize all the info
I think that if I were to build this I might skip the LLM and just setup a nice interface to the source files using qmd style search. [0]
That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.
I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.
[0] https://github.com/tobi/qmd
Awesome, I didn't know qmd style search, thank you!
I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)
0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...
I agree with the OP; this technology does not seem an appropriate fit for survival equipment. By definition survival scenarios can be life and death. Even one "You're absolutely right!" level mistake from an LLM could give a user bad information, leading to John Doe eating a mushroom he really should not have eaten.
I agree with your point, but if John Doe is foraging mushrooms without experience of the local flora (which I doubt this archive contains), then John Doe is kinda asking for it.
The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.
It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.
Decent LLMs can run on anything now: https://github.com/he-be/turbo-fieldfare
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
It already is a local archive that can be searched independently, but the LLM does RAG and is, according to them, 95% accurate. It can be useful if you don’t remember where to find a source/what words to search for, but even a durable device with Kiwix would be useless in a a lot of survival situations people imagine.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.
> Mac or PC with 16–31 GB | IBM Granite 4.2 8B | 14 of 22 right | 6 GB
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Honestly, just buy some books.
Or just.. download the books, and slap a search engine on top of them. It'll be like 10x smaller than the model. "Survival LLM" is absolutely a non-relevant use case.
Granite is not great at any size. Wonder why they picked that.
Given the whole thing reeks of vibe I don’t think anyone picked anything.
What is your favorite book on emergency appendectomies?
We had a set called something like “How to do anything in an emergency” which had instructions for this and more. Alongside lots of creative uses of potassium permanganate.
I’m pretty sure it also told you how to butcher a sheep.
I used to be obsessed with it as a kid. The spectre of nuclear war and a three volume handbook on how to survive the impending apocalypse with nothing more than a plastic bucket, a packet of potassium permanganate, some charcoal and string. The late 80s and early 90s were a wild time to be a kid.
This could be made a lot more practical (and safer) if made searchable via a simple vector search / embedding that can run on any ordinary computer.
The source (data collection) seems the most important feature here worth reusing.
No thanks, I'll use a generic model along with trusted sources.
Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.
Back in 2018 I was working on a project for educating schools in certain parts of Africa and Middle East, on AI. The issue was it had to be air-gapped (no internet guarantees) and small/portable. I ended up downloading Wikipedia (not all of it), and then created a lightweight inverted index (for search) and TextRank (for summaries). The whole thing was on a Raspberry Pi, and installed in a small plastic enclosure. We had many of these and could easily ship them. There was an "AI Learning Module" as well we built, that started with Scratch, and then moved into Python. But if I was to do it all over today, I would probably add a tiny LLM and maybe a small ASR and a TTS model too, and you basically have a completely self-contained AI knowledge assistant in a cigarette box. The point being, you basically also have a "survival AI" you can take with you. Of course you still need electricity...
I made it a while ago now, but similar: https://github.com/x3haloed/eowkit
Helps you download a local LLM and offline Wikipedia. Probably easier to just ask your favorite agent to glue something together yourself these days.
I like how both these projects seem to imply that in the event of an EOW scenario linux users can just f##k right off, thank you very much. xD
The required specs seemed a bit high to me, and I was just looking for alternatives.
I found Project Nomad (https://projectnomad.us) which looks more polished. Any other suggestions?
Love this. And as a trend, I'd love to see more local/small task-specific models/agents. Experts at helping you with that one thing, leaving the everything-and-the-kitchen-sink to the frontier models/agents
I like to use llamafile for cases like this: https://github.com/mozilla-ai/llamafile
pretty cool I like it. could you support more (small) models in the future, e.g. from the qwen or gemma family?
On the setup process: "You only click for about 5 minutes; the rest is downloading." On the energy consumption: "A small generator or power station runs it easily"
Five minutes of clicking and a (small) power station is a small price to pay when survival could be at stake.
Nice idea! I wonder how the installed IBM Granite 4.2 8B LLM does on machines with "modest" RAM
With all due respect: why does this need an LLM, and this much data? Surely a "classic" knowledge base with a more traditional search bar, running on top of the same data this LLM was trained on would be way smaller, and actually as trustworthy as the source material it surfaces for you?
There are a fair number of people who seem to believe that LLMs contain "world knowledge" and are blissfully unaware of how inaccurate an ungrounded chatbot can be. They ask it questions, it gives them confident and plausible (sometimes even correct!) answers, and they leap to the conclusion that these are useful tools for end-of-civilization knowledge bases. I fought this battle countless times on HN and Reddit, generally unsuccessfully.
This project at least attempts to ground its answers with (presumably) human-vetted sources. That will make it much, MUCH more accurate and truthful than a bare model. But there is still a non-zero chance for it to hallucinate answers, especially when posed a question that it doesn't have any source material in its database.
As for why bother hooking up an LLM instead of a simple search engine, search engines don't synthesize data from multiple sources into a coherent answer for you, but LLMs are (mostly) great at that.
That said, while I like this pattern (LLM + knowledge base) for lots of things, this isn't necessarily something I'd trust with my own (or my family's) survival.
It's disappointing that Gemma E4B seems to perform poorly on this (42% accuracy according to the GitHub). It would be nice if a $25 phone could be loaded up with this and stuffed in a EMP proof metal box with a small solar charger. Maybe a finetune or Gemma 5 will get us there.
If you're actually offline and need information in a 'survival' situation, I'm not so sure the hallucinatory nature of an LLM is the based way to distribute a repository of information. Additionally, there's no need to fit an LLM entirely in memory.
In a theoretical scenario where the internet is gone and isn't coming back: If you can deal with a very low token/second rate there's no reason not to store 'offline' a much larger model like qwen 3.8-flash-next or deepseek v4 flash 1017 and use an inference engine which can stream-read it off disk to extract the information you want, albeit slowly. Of course also download and use things that'll fit in RAM, sure, like the qwen 3.8 27B in the linked page. But no need to restrict yourself to only smaller models.
it would be useful if it would shrink a little bit
How on earth are you going to operate this if you have no electricity?