model-inference

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM