Google Cloud launches llm-d 1.0, delivering cache-aware routing for TPU and GPU clusters.
Google releases llm-d 1.0 at the PyTorch Conference North America. This is the first stable version of its open-source distributed inference stack, integrating vLLM or SGLang model servers into a Kubernetes-native unified resource pool.
Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open AIHOT original