Beyond Compression Ratio: Service-Aware KV Cache Compression for Distributed LLM Serving

Talk overview

This workshop presents KVServe's service-oriented approach to KV-cache compression, where end-to-end serving performance matters more than compression ratio alone. It covers faster compression, integration with vLLM and Mooncake, and deployment across heterogeneous platforms.