Quantum-noise-limited optical neural networks operating at a few quanta per activation