llama-cpp

DocMind AI is a powerful, open-source Streamlit application leveraging LlamaIndex, LangGraph, and local Large Language Models (LLMs) via Ollama, LMStudio, llama.cpp, or vLLM for advanced document analysis. Analyze, summarize, and extract insights from a wide array of file formats, securely and privately, all offline.

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

The self-hosted AI workflow automation platform for Git repositories.

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

InferrLM - On-device AI for iOS & Android

InsightSolver: Colab notebooks for exploring and solving operational issues using deep learning, machine learning, and related models.

Local Qwen 3.8 27B uncensored Q4_K_M + harvested SYSTEM pack. Official 3.8 weights, not a 3.6 retitle.

📚 Local PDF-Integrated Chat Bot: Secure Conversations and Document Assistance with LLM-Powered Privacy

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

Local diagnostic CLI for NVIDIA DGX Spark (GB10). Detects power caps, UMA pressure, thermal risk, CUDA 13/SM_121 wheel mismatches, Docker issues, and validates vLLM/Ollama/llama.cpp/SGLang recipes.

Self-hosted AI workspace where chat becomes visual workflows, multi-agent operations, and reviewable automations. Local memory; local or cloud models

152 open-source tools to run LLMs 100% locally – no cloud, no API keys, no censorship

Selfhost modern LLM stacks. Run the whole fleet from your terminal

A high performance C++ SDK for AI Agents

llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.

Langport is a language model inference service

On-device AI meeting assistant — real-time transcription & LLM insights, 100% offline

Fully local voice AI for iOS

Gemma models running 100% local on an 8 GB Jetson Orin Nano: voice assistants with RAG, multi-agent NPCs, live EN-JA translation, and a Gemma 4 voice + vision agent that decides when to look through the webcam. The demos from the Gemma 2 launch at Google Tokyo.

Off Grid AI — private, on-device AI. Run open models (text, vision, image, voice) locally through one OpenAI-compatible gateway. No cloud, no accounts, no API keys.