gptq

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

Large Language Models for All, 🦙 Cult and More, Stay in touch !

🦖 X—LLM: Cutting Edge & Easy LLM Finetuning

Run any Large Language Model behind a unified API

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

A guide about how to use GPTQ models with langchain

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API