gpu

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

A curated list of tools, frameworks, and resources for running AI locally — inference, RAG, orchestration, monitoring, and more.

Official GitHub repository for Phison aiDAPTIV. NAND cache memory-boosted AI inference and LLM fine-tuning platform.

I'm sorry, Dave. I'm afraid I can't let that spam call through. — Local AI answering service with voice cloning, call recording, and sub-second latency. Runs entirely on your GPU. No cloud. No per-minute billing. Just HAL.

MLOps — vLLM + LiteLLM + Portainer CE + Netdata. Case study song ngữ: kiến trúc, metrics, bài học vận hành. · Self-hosted LLM inference done the MLOps way, a bilingual case study

Remove Backgroud from a video 🧤🎥

Hands-on GPU/HPC infrastructure operations: K8s GPU scheduling, HAMi sharing, Slurm, observability & vLLM inference. Learn it free on a laptop; validate on one cheap GPU.

TokenPowerBench: Benchmarking the power consumption of LLM inference across engines and deployment configurations

Turn any NVIDIA GPU into a local AI platform. Inference + fine-tuning in your browser. One command to start, automatic clustering.

Know your VRAM before you run. Instant GPU memory estimates for any HuggingFace model.

A modular, multilingual, and multimodal Retrieval-Augmented Generation (RAG) system tailored for the financial analysis of Public Investment Fund (PIF) annual reports.

Honeycomb Lab — hex map + OpenAI gateway control plane for a home AI fleet

GPU Service Manager for LLM workloads on Linux/NVIDIA systems.

AI hardware fit calculator for LLM, embedding, reranker, OCR and VLM workloads — VRAM, throughput, licensing and multi-GPU planning

Low-Resource LLM Fine-Tuning using Unsloth and LoRA/QLoRA (WSL2/Linux) 🧠

Collection of ChatGPT alternatives & LLM tuning methods

BobVLM – A 1.5B multimodal model built from scratch and pre-trained on a single P100 GPU capable of image descriptions and moderate question answering. 🤗🎉

nvtop for vLLM — an interactive terminal dashboard for vLLM serving performance (concurrency, throughput, cache & KV memory, latency, spec-decode, GPU)

Terraform setup for deploying a private coding LLM on Vast.ai with vLLM, Qwen3 Coder, and OpenCode.

Extend LLM context windows beyond GPU memory limits with disk-backed KV cache.