GRATING: Low-Latency and Memory-Efficient Semantic Selection on Device