lmcache

CacheRoute is an innovative LLM scheduling scheme dedicated to enabling flexible KV cache reuse across LLM systems, improving task performance and system efficiency.

High-Performance KV Cache Storage Engine on CXL Shared Memory for LLM Inference