I got the complete 2.8T-parameter Kimi-K3 (~1.42 TiB) running locally on a 64GB Mac mini using MLX/Metal and SSD streaming.
SSD streaming is very slow (s/token, not tok/s)
Why would anyone (practically speaking) want this? Consider:
You have very confidential data you want analyzed by an expert, for example:
o Your doctor (and they can’t see you for three weeks)
o Attorney needs two weeks for full review of tradesecrets
So my mac-mini can spend 3-5days with Kimi-k3 and do the analysis.
I tested three with synthetic confidential-documents. Kimi-K3 recovered the planted conclusions, including overlooked invention concepts in the R&D case.
I got the complete 2.8T-parameter Kimi-K3 (~1.42 TiB) running locally on a 64GB Mac mini using MLX/Metal and SSD streaming.
SSD streaming is very slow (s/token, not tok/s)
Why would anyone (practically speaking) want this? Consider:
You have very confidential data you want analyzed by an expert, for example: o Your doctor (and they can’t see you for three weeks) o Attorney needs two weeks for full review of tradesecrets
So my mac-mini can spend 3-5days with Kimi-k3 and do the analysis.
I tested three with synthetic confidential-documents. Kimi-K3 recovered the planted conclusions, including overlooked invention concepts in the R&D case.