When can weak latent factors be statistically inferred?