load-balancer

Local AI load balancer for Ollama and MLX fleets. Auto-discovery, smart routing, OpenAI-compatible API, zero config. Perfect for Mac Minis & Studios.

TokenFlow is an enterprise-grade, token-aware load balancer and AI gateway for vLLM. It prevents GPU OOM crashes using Redis-backed admission control, semantic caching, and dynamic token routing. Built in TypeScript for extreme bare-metal performance.