> Anything that suggests curl into bash just plain sketches me out.
Yeah, 100% and it's becoming more and more of a thing, see rust install for example.
OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on you Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
One way I prevent possible catastrophic fuckups is that the 'doing code work' box that runs opencode or pi or whatever, is its entirely own separate VM and desktop environment (running as a xen or kvm guest), than the machine running llama-server itself.
The harness gets the openai-compatible endpoint fed into it to talk to llama-server across the network, but the VM has no access whatsoever to my personal files, mail, backups/deep storage, fileserver, Documents folder, etc.
I think I can probably run Gemma 3 12B on my macbook M3 pro with 18GB. The question is, should I do it? This small model is probably not capable of doing a lot or advanced coding or reasoning. What else could it be used for, since it can run locally and privately?
I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.
I'm getting roughly twice the pp speed using ik_llama fork with a Cascade Lake Xeon, but their MTP implementation seems faulty compared to llama.cpp so token generation is often slower. Can't have nice things.
Anything that suggests curl into bash just plain sketches me out.
Git clone llama.cpp and build it, it's not hard.
https://github.com/ggml-org/llama.cpp/blob/master/docs/build...
literally just a few steps for the basics:
git clone https://github.com/ggml-org/llama.cpp
cmake -B build
cmake --build build --config Release
> Anything that suggests curl into bash just plain sketches me out.
Yeah, 100% and it's becoming more and more of a thing, see rust install for example.
OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on you Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
One way I prevent possible catastrophic fuckups is that the 'doing code work' box that runs opencode or pi or whatever, is its entirely own separate VM and desktop environment (running as a xen or kvm guest), than the machine running llama-server itself.
The harness gets the openai-compatible endpoint fed into it to talk to llama-server across the network, but the VM has no access whatsoever to my personal files, mail, backups/deep storage, fileserver, Documents folder, etc.
I think I can probably run Gemma 3 12B on my macbook M3 pro with 18GB. The question is, should I do it? This small model is probably not capable of doing a lot or advanced coding or reasoning. What else could it be used for, since it can run locally and privately?
I was a bit suspicious of the url but it is also listed on llama.cpp github
https://github.com/ggml-org/llama.cpp
Same, but it looks legit enough to me. Here is the git repo for the site with a link back to llama.app: https://github.com/ggml-org/llama.pages
(I still deeply distrust curlpipes in general though.)
ty for digging this up!
I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.
the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.
I'm getting roughly twice the pp speed using ik_llama fork with a Cascade Lake Xeon, but their MTP implementation seems faulty compared to llama.cpp so token generation is often slower. Can't have nice things.
laama.ccp