Designing Neural Synthesizers for Low Latency Interaction