Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models