vllm-plugins

Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM

Production inference for encoder models - ColBERT, GLiNER, ColPali, embeddings etc. - as vLLM plugins for online and in-process deployment

Proxima lets existing GPUs serve 4x more concurrent requests

An out-of-tree vLLM plugin for Mobilint NPU runtime integration.

FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference