Unifying Model and Layer Fusion for Speech Foundation Models