Revealing Secrets From Pre-trained Models