open-source-ai• 15 min
Ray Serve: A scalable model serving library (Apache 2.0, 35k stars)
A scalable model serving library built on Ray for distributed inference with autoscaling, batching, and multi-model pipelines.
#ray-serve#ray
#autoscaling
1 post