ray-serve

An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale

This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.

Self-hosted, OpenAI-compatible inference for the agentic era: reasoning LLMs, universal tool calling, and the Responses API alongside embeddings, speech, and image models — many models sharing your GPUs, one gateway. Powered by Ray Serve.

Batch LLM Inference with Ray Data LLM: From Simple to Advanced