Tsinghua and CUHK team open-source AKV: An engineering service system for KV cache in LLM agents
Researchers from Tsinghua University and CUHK have open-sourced AKV, a KV cache engineering service system built on SGLang. This allows agents to discard or rearrange KV entries via Drop and Repos (RoPE rotary repositioning) operations without rewriting the text history.
Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open AIHOT original