vLLM Workshop
llm-d: A Distributed Inference Plane Across Workloads, Modalities, Hardware, and Platforms
Date, time, and room will be added once confirmed.
Talk overview
This talk introduces llm-d's distributed inference control plane, including intelligent routing, endpoint selection, inference pools, model servers, prefix- and load-aware scheduling, hierarchical KV-cache offloading, Prefill–Decode disaggregation, expert parallelism, flow control, and elastic scaling across Kubernetes, Slurm, Ray, and bare metal.
