asprenger/ray_vllm_inference

A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.

⭐ 79 ⑂ 13 Python Apache-2.0 · 2024-04-07推送

inference llm llm-serving llmops mlops model-serving pytorch ray transformer vllm

79

Watchers

0

贡献者

0

Commits

0

Releases

4

Open Issues

2024-04-07

最近推送

原文中文

暂无 README