gpu

GPU worker client for the Talos network. Pairs with your Talos account, serves open-model inference jobs over a WebSocket, and reports uptime for payouts.

GPU environment and cluster management with LLM support

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

Collection of best practices, reference architectures, model training examples and utilities to train large models on AWS.

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

RAG (Retrieval-augmented generation) ChatBot that provides answers based on contextual information extracted from a collection of Markdown files.

A work in progress to build out solutions in Rust for MLOPs

Vector search engine inside Milvus, integrating FAISS, HNSW, DiskANN.

Persist and reuse KV Cache to speedup your LLM.

Easy-Translate is a script for translating large text files with a SINGLE COMMAND. Easy-Translate is designed to be as easy as possible for beginners and as seamlesscustomizable and as possible for advanced users.

🚀 200倍速！AI时代的下载神器 | Docker/PyPI/HuggingFace/CRAN 全加速 | 并行分片+智能缓存，让下载飞起来

Local LLM Testing & Benchmarking for Apple Silicon

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

Plug-and-play homelab dashboard in one container — GPU, local-AI VRAM, Docker, systemd, host health. Built-in read-only MCP server so AI agents can explore it too.

Ollama Cloud is a Highly Scalable Cloud-native Stack for Ollama

GGNN: State of the Art Graph-based GPU Nearest Neighbor Search

A high-performance RL training-inference weight synchronization framework, designed to enable second-level parameter updates from training to inference in RL workflows

RapidFire AI: Rapid AI Customization from RAG to Fine-Tuning

A manual for helping using tesla p40 gpu