cpu-inference

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

Speaker diarization for Python — "who spoke when?" CPU-only, no API keys, Apache 2.0. ~10.8% DER on VoxConverse, 8x faster than real-time.

Sparse-upcycle a dense Gemma-3-270m into a Mixture-of-Experts (gemma3moe) and serve it on llama.cpp (CPU). Upcycle, train, inspect routing, GGUF.

🔥 Alternative to Ollama — multi-model serving with sub-ms model switching · CPU-only 20B inference for Edge AI · llama.cpp + stablediffusion.cpp

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig

Lightning-fast RAG for AI agents. ONNX-powered, 4-layer fusion, MCP server. No PyTorch.