llm-db/FineInfer

Deferred Continuous Batching in Resource-Efficient Large Language Model Serving (EuroMLSys 2024)

⭐ 19 ⑂ 2 Python MIT · 2024-05-29推送

https://dl.acm.org/doi/10.1145/3642970.3655835

fine-tuning inference llm lora peft pytorch co-serving

19

Watchers

0

贡献者

0

Commits

0

Releases

0

Open Issues

2024-05-29

最近推送

原文中文

暂无 README