Star 总榜

在nano-vllm基础上支持moe模型以及Speculative Decoding技术

GLM-5.2 QuantTrio (Unpruned ) on 4xDGX Spark (GB10): 327K-655K context or up to 5 concurrent agents, one receipe. TP4 + DCP + MTP

Extend LLM context windows beyond GPU memory limits with disk-backed KV cache.

Kubernetes-native control plane for scale-to-zero serving of long-tail LLMs

Tencent Hy3 295B MoE (NVFP4) on 2x NVIDIA DGX Spark — TP2 over 200GbE, 256K context, 26 tok/s end-to-end. Tuned, benchmarked, agent-ready.

Multi-model LLM serving for NVIDIA DGX Spark with vLLM, web UI, and tool calling

Repository for Multililngual Generation, RAG evaluations, and surrogate judge training for Arena RAG leaderboard (NAACL'25)

Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies

📑 npm pacakge to Craft files into Markdown with ease

Docker Compose setup that integrates vLLM, Open WebUI and NGINX (for SSL termination).

Spark Pulse is a web control plane for spark-vllm-docker

Inference and finnetunning of a VLM (LLama Vision 11b) using the Unsloth, PEFT libraries and torch.

Qwen3.6 vLLM Toolkit — Launcher + Templates, Optimized for 2×24 GB. vLLM launcher with an optimized hybrid chat template drawing from the best community fixes for Qwen 3.6 27B.

General Information, model certifications, and benchmarks for nm-vllm enterprise distributions

The official implementation of CVPR Workshop 2025 paper: Window Token Concatenation for Efficient Visual Large Language Models.

Professional CLI toolkit for Modal GPU workflows, Docker Compose deployments, Python runtimes, model serving, and account/billing management.

GPU orchestrator with vLLM sleep mode

Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.

Reward a Language Model with pancakes 🥞

Open-source framework for turning expert knowledge into PII-free synthetic conversational data and production-ready LoRA adapters.