Kernel-based potential mean-field games with unbiased random Fourier $U$-statistics