Investigating Pre-trained Audio Encoders in the Low-Resource Condition