← 返回专题广场
kubernetes-operator
6 个项目 · ⭐ 18.1k3
13 小时前
最近推送
4
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Go
⭐ 196
⑂ 30
Apache-2.0
· 4 小时前推送
4 小时前
最近推送
6
2024-04-11
最近推送