Last updated: 11/9/2026Last verified: 2026-11-07
<p>kimi-k3-in-c is a portable C99 inference engine that streams the trillion-parameter Kimi K3 checkpoint from disk, so the same 2.78T-parameter model runs on one CPU with a measured 8.24 GB peak RSS and byte-identical output from 8 GB machines to 224 GB ones.</p><ul><li>Streamed MoE experts from a 1.56 TB checkpoint keep peak RSS as low as 8.24 GB with byte-identical output</li><li>Portable C99 with no GPU, no BLAS, and no framework; AVX2+FMA on x86-64 and NEON on Apple Silicon</li><li>Preset memory ladder from laptop to server where more RAM only buys speed</li><li>Weightless verification gates reproduce reference outputs; a doctor script sizes presets to your hardware</li></ul>
Details
Platform
macOSWindowsLinux
Features
- Streamed MoE experts from a 1.56 TB checkpoint keep peak RSS as low as 8.24 GB with byte-identical output
- Portable C99 with no GPU, no BLAS, and no framework; AVX2+FMA on x86-64 and NEON on Apple Silicon
- Preset memory ladder from laptop to server where more RAM only buys speed
- Weightless verification gates reproduce reference outputs; a doctor script sizes presets to your hardware
Languages
en
Best for
Local-LLM researchers studying low-RAM streaming inference without GPUsEngineers exploring MoE checkpoint streaming, expert caches, and kernel contractsEdge and on-prem builders prototyping huge-model feasibility before buying GPU capacity
Known limitations
- Checkpoint storage gate is ~1.7 TB free disk; download and trunk-packing take hours the first time
- Throughput is minutes per token on constrained-memory machines; this project prioritizes fit over speed
- Text-only chat scope today: no tools, vision/image parts, HTTP serving, or context compaction yet
- Model weights are owned and licensed by Moonshot AI; the repository ships the engine, not the weights
Alternatives
llmfit — Plan mode estimates which models fit your RAM, but does not itself run inference
Unsloth Desktop — GUI app for running much smaller local models comfortably
Vicuna-13B — Compact open chat model that fits entirely in RAM
