Optimization of Edge-Side Inference Performance for Open-Source Large Models and Collaboration Between Agents

Talk overview

Optimization of Edge-Side Inference Performance for Open-Source Large Models and Collaboration Between Agents With the rapid growth of edge computing, deploying open-source large language models on resource-constrained edge devices has become a critical challenge. This work focuses on optimizing edge-side inference performance through model compression, quantization, pruning, and efficient scheduling techniques, reducing latency, memory footprint, and energy consumption while preserving accuracy. Beyond single-model optimization, we explore collaboration mechanisms among multiple agents, where lightweight edge agents and cloud-based models coordinate through task decomposition, distributed inference, and knowledge sharing. Experimental results demonstrate that the proposed framework significantly improves throughput and responsiveness, enabling real-time intelligent applications such as autonomous systems, IoT, and mobile assistants in edge environments.