Inference-time sparse attention with asymmetric indexing