Datacenter-scale distributed inference serving framework that orchestrates vLLM, SGLang and TensorRT-LLM across multiple GPUs and nodes.
8.4K1.8KRustOtherUpdated 7 hours ago
#disaggregated-serving
Open-source projects tagged “disaggregated-serving”
Curated GitHub repositories related to disaggregated-serving, with AI summaries and key stats.