All roughly the same price, for a consumer who has won the lottery:
- 1x 512 GB Mac Studio with the 4 TB disk storage option.
- 4x Nvidia GB10s (total 512 GB RAM) each connected with a 100 GiB/s link to a Mikrotik CRS504-4XQ-IN.
- 3x Ryzen AI Max+ 495 192GB systems (total 576 GB RAM), connected via a 40 GiB/s link directly to each other.
Has any individual/SMB purchased this yet? Aware the ROI isn't there for most individuals that don't heavily use LLM for coding or multi agentic workflows -- but for those that it does make sense this is ~$68k cheaper than the next alternative making this the most realistic option for consumer grade access to frontier models.
Specifically interested if anyone has ran GLM-5.3 (753B MoE) in 4-bit on a 512GB Mac Studio. Looking for real world usage numbers and/or any benchmarks you might have performed.
GLM-5.3 would be bottlenecked by the M5's GPU, here. I don't have any hard numbers, but Apple Silicon is known for weak prefill performance on larger models (yes, even MoEs).
Good to know. Private GLM-5.3 is really the one use-case I can imagine for a system like this - It does very well with cybersecurity tasks, which are either blocked for most people by current providers, or on their way to getting blocked based on the regulatory/political trends. Such a capability could become very valuable when supply becomes artificially limited, although I expect the Chinese labs will be the last hold-outs. Imagine not being able to do security hardening on your own tech because government-imposed guardrails decide your legitimate prompts appear "risky". In such a world, what would an unconstrained cyber-capable model be worth?
All roughly the same price, for a consumer who has won the lottery: - 1x 512 GB Mac Studio with the 4 TB disk storage option. - 4x Nvidia GB10s (total 512 GB RAM) each connected with a 100 GiB/s link to a Mikrotik CRS504-4XQ-IN. - 3x Ryzen AI Max+ 495 192GB systems (total 576 GB RAM), connected via a 40 GiB/s link directly to each other.
When the page first loads, it shows a different price for ~1sec. It quickly overwrites that and displays the much larger, eye watering figure.
1st load ref: https://i.postimg.cc/YC5V5yZH/prerewrite.png
Dupe/Discussion (67 points, 1 week ago, 82 comments) https://news.ycombinator.com/item?id=50277166
Has any individual/SMB purchased this yet? Aware the ROI isn't there for most individuals that don't heavily use LLM for coding or multi agentic workflows -- but for those that it does make sense this is ~$68k cheaper than the next alternative making this the most realistic option for consumer grade access to frontier models.
Specifically interested if anyone has ran GLM-5.3 (753B MoE) in 4-bit on a 512GB Mac Studio. Looking for real world usage numbers and/or any benchmarks you might have performed.
Thanks!
GLM-5.3 would be bottlenecked by the M5's GPU, here. I don't have any hard numbers, but Apple Silicon is known for weak prefill performance on larger models (yes, even MoEs).
Good to know. Private GLM-5.3 is really the one use-case I can imagine for a system like this - It does very well with cybersecurity tasks, which are either blocked for most people by current providers, or on their way to getting blocked based on the regulatory/political trends. Such a capability could become very valuable when supply becomes artificially limited, although I expect the Chinese labs will be the last hold-outs. Imagine not being able to do security hardening on your own tech because government-imposed guardrails decide your legitimate prompts appear "risky". In such a world, what would an unconstrained cyber-capable model be worth?
[flagged]