Hi HN — I spent the last three months building an LLM inference engine from scratch in C.
The result is *geistlib*: it runs Microsoft’s BitNet 2B model fully offline on a 4 GB Raspberry Pi 5, packaged as a single ~1.2 GB executable containing both the engine and the model.
The runtime itself is under 1 MB. The model weights are embedded directly in the executable and demand-paged, so memory usage is comparable to mmap'ing a separate model file.
This is not a llamafile wrapper around llama.cpp. The inference engine is written from scratch and specialized for BitNet's ternary weights: −1, 0, +1.
On ARM64, that means the main inference loop can replace much of the usual multiplication work with additions and map it onto ARM's SDOT instruction.
You don't have to trust my numbers. `make bench` runs the frozen benchmark protocol locally, waits for thermal conditions, and compares against llama.cpp and bitnet.cpp when installed.
And, of course, this is still a 2B model. During testing it confidently informed me that the Moon is "60% oxygen." I kept that example in the docs.
One more disclosure: I built much of this through heavy pair-programming with Claude Code. Because of that, I tried to make correctness measurable rather than something you have to take on faith.
Happy to answer questions about the ternary kernels, embedding a 1.2 GB model directly into an executable, ARM optimization, or why I chose C instead of Rust.
Hi HN — I spent the last three months building an LLM inference engine from scratch in C.
The result is *geistlib*: it runs Microsoft’s BitNet 2B model fully offline on a 4 GB Raspberry Pi 5, packaged as a single ~1.2 GB executable containing both the engine and the model.
Install and run:
``` curl -L -o geist-bitnet https://github.com/geisten/geistlib/releases/latest/download... chmod +x geist-bitnet ./geist-bitnet "The three largest moons of Jupiter are" ```
The runtime itself is under 1 MB. The model weights are embedded directly in the executable and demand-paged, so memory usage is comparable to mmap'ing a separate model file.
This is not a llamafile wrapper around llama.cpp. The inference engine is written from scratch and specialized for BitNet's ternary weights: −1, 0, +1.
On ARM64, that means the main inference loop can replace much of the usual multiplication work with additions and map it onto ARM's SDOT instruction.
On my Raspberry Pi 5:
* geistlib: *15–18 tokens/s* * Microsoft bitnet.cpp: *9.3 tokens/s* * Energy: *0.44 J/token*
Same board, same model.
Benchmarks, recordings, methodology, and energy measurements:
https://github.com/geisten/geistlib/blob/main/docs/DEMOS.md
https://gist.github.com/geisten/a5cb278d2a61d88c7eb82f6baec8...
You don't have to trust my numbers. `make bench` runs the frozen benchmark protocol locally, waits for thermal conditions, and compares against llama.cpp and bitnet.cpp when installed.
And, of course, this is still a 2B model. During testing it confidently informed me that the Moon is "60% oxygen." I kept that example in the docs.
One more disclosure: I built much of this through heavy pair-programming with Claude Code. Because of that, I tried to make correctness measurable rather than something you have to take on faith.
The project is Apache-2.0:
https://github.com/geisten/geistlib
Happy to answer questions about the ternary kernels, embedding a 1.2 GB model directly into an executable, ARM optimization, or why I chose C instead of Rust.