paged-attention

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

A High-Performance LLM Inference Engine with vLLM-Style Continuous Batching

A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM