quantization

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Official code for the Manning book on structural LLM optimization: depth/width pruning, knowledge distillation, and attention optimization, runnable on free Colab GPUs.

Nodes to run Hunyuan Image 3 locally with BF16 and NF4 quantized options in Comfyui

A native macOS lab for teaching tiny language models to think — build the architecture, train the weights, and watch a small LLM emerge from scratch, locally on Apple Silicon with custom data, tokenizers, checkpoints, and MLX acceleration.

Open source skill library for AI coding agents to write, optimize, and debug high performance compute kernels across CUDA, Triton, and quantized workloads.

Wav2vec 2.0 Self-Supervised Pretraining

Faiss-based library for efficient similarity search

Open-source platform for creating, distributing and running sovereign EU-compliant LLMs. Verticalize any model for your domain, language and brand. AI Act ready.

Quantized stable-diffusion cutting down memory 75%, testing in streamlit, deploying in container

Code for Deep Learning for Modern AI

Fastest Diffusion backend, WebUI, server. Pushing implementation and discovery of optimizations techniques 🚀

🏆Winning Project | ModelGate is a contract-aware AI control plane that ingests customer contracts, extracts SLA/privacy/routing constraints, and generates an OpenAI-compatible endpoint that automatically routes every request to the optimal model. Simple queries go to cheap models. Complex queries go to premium ones.

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

reference pytorch code for intent classification

First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

Curated list of methods that focuses on improving the efficiency of diffusion models

Native Windows vLLM 0.27.1 wheels: Python 3.13, PyTorch 2.13 + CUDA 13.0, SM 7.5-12.0 for RTX 20/30/40/50, OpenAI-compatible serving, FlashAttention/Rust, 10 KV formats, Multi-TurboQuant, and experimental CPU/RAM/NVMe prompt-KV offload - no WSL or Docker.

A guide about how to use GPTQ models with langchain

Run, quantize, and fine-tune LLMs on Apple Silicon. Pure Rust, no Python, no CUDA, no ONNX

Fine-tune Mistral 7B to generate fashion style suggestions