NN-ResDMD: Learning Koopman Representations for Complex Dynamics with Spectral Residuals