ggml

Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

Go manage your Ollama models

ggml speech-to-text inference for 16+ model families

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose

Self-evaluating interview for AI coders

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)

CLIP inference in plain C/C++ with no extra dependencies

C++ ggml runtime hub for multilingual ASR and TTS models: Cohere Transcribe, Parakeet TDT, Voxtral, Canary 1B v2, etc, plus universal forced alignment, and more

Large Language Models for All, 🦙 Cult and More, Stay in touch !

audio.cpp with a full-task WebUI - pure C++ audio-model inference engine powered by ggml. TTS, ASR/STT, VAD, voice conversion, speaker diarization, music generation. No Python dependency.

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

Local-first speech-to-text: no cloud, no telemetry, fail-closed by design. One CLI, seven model families, signed model catalog, OpenAI-compatible local API.

Chat with your data privately using MPT-30b

True on-device AI for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM, Speech-to-Text and Image Generation — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

stable-diffusion.cpp bindings for python