vllm

Kubernetes-native control plane for scale-to-zero serving of long-tail LLMs

Tencent Hy3 295B MoE (NVFP4) on 2x NVIDIA DGX Spark — TP2 over 200GbE, 256K context, 26 tok/s end-to-end. Tuned, benchmarked, agent-ready.

Multi-model LLM serving for NVIDIA DGX Spark with vLLM, web UI, and tool calling

Repository for Multililngual Generation, RAG evaluations, and surrogate judge training for Arena RAG leaderboard (NAACL'25)

Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies

📑 npm pacakge to Craft files into Markdown with ease

Docker Compose setup that integrates vLLM, Open WebUI and NGINX (for SSL termination).

Spark Pulse is a web control plane for spark-vllm-docker

Qwen3.6 vLLM Toolkit — Launcher + Templates, Optimized for 2×24 GB. vLLM launcher with an optimized hybrid chat template drawing from the best community fixes for Qwen 3.6 27B.

General Information, model certifications, and benchmarks for nm-vllm enterprise distributions

The official implementation of CVPR Workshop 2025 paper: Window Token Concatenation for Efficient Visual Large Language Models.

Professional CLI toolkit for Modal GPU workflows, Docker Compose deployments, Python runtimes, model serving, and account/billing management.

Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.

Inference and finnetunning of a VLM (LLama Vision 11b) using the Unsloth, PEFT libraries and torch.

GPU orchestrator with vLLM sleep mode

A voice assistant with local LLM as a backend

Read PGS (Bluray) and VobSub (DVD) image subtitles and extract their text using external Vision Language Models.

Lna-Lab production pipeline: GGUF -> modelopt-format NVFP4 + working MTP head for vLLM on RTX PRO 6000 Blackwell (SM120). Stages 2 (NVFP4) and 3 (MTP graft) are Lna-Lab originals; stage 1 (GGUF->bf16) reuses li-yifei/gguf-to-nvfp4.

SkyPortal SDK & CLI - Open Source

TokenFlow is an enterprise-grade, token-aware load balancer and AI gateway for vLLM. It prevents GPU OOM crashes using Redis-backed admission control, semantic caching, and dynamic token routing. Built in TypeScript for extreme bare-metal performance.