EP329: Teaching AI to forget the right things

Title: Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLMSource: http://arxiv.org/abs/2606.29563v1Summary:This paper introduces K-VEC, a coverage-aware KV-cache eviction strategy that resolves performance drops in long-context tasks by maximizing token retention across attention heads and layers. By theoretically linking token coverage to mutual information, it provides a foundational efficiency breakthrough for deploying long-context LLMs under resource constraints.