vllm

Uncensored AI platform — chat, roleplay, image generation. Built on open-source models.

MLOps — vLLM + LiteLLM + Portainer CE + Netdata. Case study song ngữ: kiến trúc, metrics, bài học vận hành. · Self-hosted LLM inference done the MLOps way, a bilingual case study

Source code for the paper: Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention

Drop-in OIDC & Google A2A auth + Weaviate memory for Ollama, vLLM and any local LLM server.

Agentic SDLC with local LLM's.

Dual-mode cloud burst LLM router. Edge-first or cloud-first inference with data sovereignty, cost controls, and automatic failover. https://aiburstcloud.com

Convert pdf and image files into markdown

Local-first AI coding agent runtime — your models, fail-closed safety, evidence-gated self-improvement

vLLM + Qwen3.6-27B (BF16) OpenAI-compatible inference server on AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). Vision input, 256K context, /v1/responses with separated reasoning, via TheRock ROCm.

The official repo for the paper "Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching"

KONASH: Train knowledge agents that search, retrieve, and reason. Based on KARL (Databricks, 2026).

The official implementation of NeurlPS 2025 D&B paper: IndustryEQA: Pushing the frontiers of Embodied Question Answering in Industrial Scenarios.

Diagnose vLLM inference servers

Interested in running a conspiracy of agents (technical term) on your Local AI Infra? Who isn't!

Local LLM speed measurements from a 4x RTX 3090 rig, with exact launch commands and per-run provenance.

Hands-on GPU/HPC infrastructure operations: K8s GPU scheduling, HAMi sharing, Slurm, observability & vLLM inference. Learn it free on a laptop; validate on one cheap GPU.

🚀🛸 Easily boost the speed of pulling your models and datasets from various of inference runtimes. (e.g. 🤗 HuggingFace, 🐫 Ollama, vLLM, and more!)

Thinking-native compiler for LEET Language Understanding item sets — killer-item design grammar, verification separation, deterministic gate chain, 3-model benchmark

Source-code level analysis of LLM RL training infra: async RL, weight sync, FP8, MoE routing | LLM RL 训练基础设施源码级分析

AI-Inference-Managed-by-AI: Go binary for managing AI inference on edge devices