When the KV Cache Cannot Roll Back
How a phone skips re-reading a 2,050-token agent prompt, built twice because the app's two runtimes have different cache constraints.
Abstract
Prefix reuse on Android, presented with Arjhine Ty. Reusing 1,192 tokens takes Qwen3-1.7B on ExecuTorch from a 5,852 ms first turn to 215 ms on the second.
Upcoming
- PyTorch Day JapanTokyo, 10 December 2026