open-source-ai• 15 min
Text Generation Inference: Hugging Face's production-grade LLM inference server (Apache 2.0, 9k stars)
Deploying LLM inference at scale with continuous batching, tensor parallelism, and flash attention — Hugging Face's production-grade inference server.
#tgi#huggingface