#ray

Ray Serve: A scalable model serving library (Apache 2.0, 35k stars)

A scalable model serving library built on Ray for distributed inference with autoscaling, batching, and multi-model pipelines.