flash-attention

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

MoBA: Mixture of Block Attention for Long-Context LLMs

Emotion text classification using Llama3-8b with LoRA and FlashAttention. Based on LLaMA-Factory.

Native Windows vLLM 0.27.1 wheels: Python 3.13, PyTorch 2.13 + CUDA 13.0, SM 7.5-12.0 for RTX 20/30/40/50, OpenAI-compatible serving, FlashAttention/Rust, 10 KV formats, Multi-TurboQuant, and experimental CPU/RAM/NVMe prompt-KV offload - no WSL or Docker.

Automatically benchmark and optimize attention in diffusion models. 1.5-2x speedup on RTX 4090.

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

FlashAttention-style custom attention backend for vLLM on AMD MI50/MI60/Radeon VII (gfx906). Downstream fork of mixa3607/ML-gfx906 with replacement HIP kernels and a vllm.general_plugins entry point.

Lightweight, Self-Hosted AI Guardrails Model based on ModernBERT.