open-source-ai• 15 min
XInference: A distributed LLM inference platform (Apache 2.0, 5k stars)
Supporting 100+ models with built-in embedding, reranking, and multi-GPU inference — XInference unifies inference engines under one OpenAI-compatible API.
#xinference#distributed