Extracting and Understanding the Superficial Knowledge in Alignment