on-device-ai

Production ready toolkit to run AI locally

14MB foundation model for tiny devices; phones, wearables, smart home, and robots.

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

AI You Control: Choose your models. Own your data. Eliminate vendor lock-in.

An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.

On-device, real-time multimodal AI with features similar to GPT-Live

Core ML model zoo for iOS/macOS — PyTorch models converted to ready-to-use .mlpackage, each with a conversion script and SwiftUI sample app. Sibling repos cover Apple's Core AI framework (iOS/macOS 27) and on-device LLMs.

Declarative way to run AI models in React Native on device, powered by ExecuTorch.

Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.

PhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference.

A practical lab for building, testing, and evaluating apps with Apple's Foundation Models framework.

NativeMind: Your fully private, open-source, on-device AI assistant

PokeClaw (PocketClaw) — first on-device AI that controls your Android phone. Gemma 4, no cloud, no API key. Poke is short for Pocket.

Muesli - agent-native local meeting transcription + dictation for macOS (Granola + WisprFlow alternative)

Optimized Whisper models for streaming and on-device use

Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking

Shared Single-file memory layer for all your agents, sub mili-second RAG over text, photo and video on Apple Silicon.. No Server. No API. One File. Pure Swift