Emergent Linguistic Structures in Neural Networks are Fragile